4

I'm currently writing a service that needs to integrate with Eventbrite to store external event information as items in Sitecore. We run a Sitecron job every 4 hours to pull in and update the existing data set. This does a few checks to see whether or not the item exists, and if it does, if it needs updating. If the item is new, then a new item is created. The first check that's performed is for the venues of the events. Due to a problem with the way that Eventbrite stores venues, there are several distinct "venues" that are all the same location. With that, I normalize the venues into a separate item.

The Venues that are retrieved from the Eventbrite API are generated with this method

private void CreateVenue(IEnumerable<Venue> venues)
{
  var contextDB = Factory.GetDatabase("master");
  var _scService = new SitecoreService(contextDB);
  using (var searchContext = ContentSearchManager.GetIndex("sitecore_master_index").CreateSearchContext())
  {
    var venueResult = searchContext.GetQueryable<VenueByIdSearchResult>().Where(x => x.TemplateId == VenueByIdSearchResult.VenueTemplateId).ToList();
    var baseQuery = PredicateBuilder.True<VenueByIdSearchResult>();
    var queryByTemplate = PredicateBuilder.False<VenueByIdSearchResult>();
    queryByTemplate = queryByTemplate.Or(t => t.TemplateId == VenueByIdSearchResult.VenueTemplateId);

    foreach (var venue in venues)
    {
      var queryByVenueId = PredicateBuilder.False<VenueByIdSearchResult>();
      queryByVenueId = queryByVenueId.Or(v => v.VenueId == venue.Id);
      var existingItemQuery = baseQuery.And(queryByTemplate).And(queryByVenueId);
      var existingItem = searchContext.GetQueryable<VenueByIdSearchResult>()
          .Where(existingItemQuery);
      IVenue scVenue;
      if (existingItem.Any())
      {
        scVenue = _scService.Cast<IVenue>(existingItem.First().GetItem());
      }
      else
      {
        scVenue = _scService.Create<IVenue, IModelBase>(EventList.APIVenuesFolder, venue.Id);
      }
      scVenue = venue.MapVenue(scVenue);
      _scService.Save(scVenue);
    }
  }
}

The VenueByIdSearchResult class, which inherits SearchResultItem looks like this

private class VenueByIdSearchResult : SearchResultItem
{
  public static ID VenueTemplateId { get; } = new ID("{9DD2BE5A-BEDB-4B49-B769-EA8A1DC21D57}");
  [IndexField("eventbrite_venue_id")]
  public string VenueId { get; set; }
}

(yes, this class is currently a private class within my Service class that Sitecron is interacting with, it's a piece that will move in the near future)

The item creation steps work great, and aren't a problem (although I wish they were faster. I believe this is a side-effect of using Glass and not using a BulkUpdate context).

The biggest problem lies with the usage of PredicateBuilder to formulate the queries that are joined to find the existingItem. Locally, using Lucene as the search provider, this code works fine and the item is appropriately found. This ultimately results in an update occurring to the existing item, rather than creating a new item. When I run the exact same code, in an environment where Solr is the search provider, the existingItem is not found with the predicate from PredicateBuilder.

The other two methods that perform almost identical lookups to create/update items face the same issue. Instead of finding the item, and updating it, it just creates new items every time the Sitecron job runs. This is not ideal.

Another thing I noticed, when trying to debug this, is that in my search log, it appears when the queries for the Venue items are logged, the IndexField is not being converted appropriately.

As shown in the VenueByIdSearchResult class, the VenueId field is using an IndexFieldAttribute to map the indexed field eventbrite_venue_id, which does exist in my Solr index as eventbrite_venue_id_t, which I would expect. Instead the query looks like this.

(_template:(9dd2be5abedb4b49b769ea8a1dc21d57) AND eventbrite_venue_t:(26990187))

When I use this exact query in my Solr admin, I don't get any results (obviously) but if I change eventbrite_venue_t to eventbrite_venue_id_t in the query, it does exactly as I'd expect. I'm unsure where the Search.log query is coming from, or how that field is being changed from what the IndexFieldAttribute is setting it to.

Ultimately, my understanding was that Content Search API was intended to abstract away the Solr vs Lucene vs Coveo vs whatever discrepancies to maintain consistency. I apologize if my understanding is incorrect, and would love to see how I'm misunderstanding it.

For final context, this is Sitecore 8.2u2/Solr 5.5.3 in a distributed set up.

Please let me know if I need to provide additional information.

UPDATE

So, I tried installing Solr locally to see if I could reproduce the problem. At first, I tried just migrating over to my Bitnami install of Solr 5.5.0-0. After some issues, and realizing that there is an open issue with version of Solr<5.5.1, I installed Bitnami's Solr 6.0.1 instance. Once doing that, and migrating my cores over to there, and rebuilding, I was able to re-run the above batch locally. It worked. The queries look exactly as I'd expect, and there were no data replication issues. At this point, I'm curious if there are any issues between Solr 5.5.3 (the version on my integration server) and Solr 6.0.1. Any information would be greatly appreciated.

Update 2 Turns out the problem is a little more interesting. We recently upgraded to Sitecore 9.0.2 and the issue came back. Fields that are indexed as "eventbrite_venue_id_t" are being translated to "eventbrite_venue_t" when the Indonesian language translations are in the solr managed-schema. I believe this is a bug with IndexFieldAttribute and will be following up with Sitecore. It appears that any template that is named with "id" at the end of the name will face this issue. I haven't tested with other languages, but I suspect they would face the same issue.

5
  • Within your solr default configuration from showconfig, do you have venueId being mapped to another name. Can you try to add the entry <fieldType fieldName="eventbrite venue id" returnType="string"/> in the xpath sitecore/contentSearch/indexConfigurations/defaultSolrIndexConfiguration/fieldMap/fieldNames (raw:AddFieldByFieldName). I've not used Glass Mapper and it may change to xpath. You may need to update accordingly for the raw:AddFieldByFieldName Feb 23, 2019 at 19:29
  • Hey Hishaam, I've not added any custom fieldtypes, so no none of the fields I'm trying to work with are in the defaultSolrIndexfConfigruation. I thought that template fields were indexed automatically, if they were being queried from one of the default indexes Feb 23, 2019 at 20:24
  • My advice is to be on the highest supported version, which for 8.2.2 seems to be Solr 6.1 kb.sitecore.net/articles/227897. If possible I'd advise updating to 8.2.7+ so you can use Solr 6.6.3. Being on the highest supported version will greatly reduce the chance of these kind of issues. Feb 24, 2019 at 1:45
  • @RyanShohoney If your issue is resolved, please post the updates as the accepted response, so that the question does not show up as unanswered. This will also help the site get out of beta by raising our answered question ratio.) Thanks! Feb 27, 2020 at 16:00
  • @DanSolovay, I'll mark it as answered as soon as I hear back from Sitecore about whether or not they view it as a bug. If they don't view it as a bug, there will be a different resolution, which I'll post as an actual answer. Feb 27, 2020 at 17:35

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.