3
  • SC 9.1
  • SwitchOnRebuildSolrSearchIndex is enabled
  • SOLR 7.2.1

I have 120k items in Sitecore, so I have a good bit of data to sample. When I rebuild my web index, the total number is consistent. But when I do a query like this to get my events, it will says 5382.

https://solr:8983/solr/sitecore_web_index/select?q=_template:288a9dfa5ebe41f0b21bf20bd97a5d2c

I can rebuild the index and now it says 4924. If I look at the secondary index and it says 5382 just like it used to before the index was switched out for the new one.

I can rebuild the index 40 times and get 30 different results. When I look in the logs I have 4 item errors relating to hexidecimal data. I can do this across all my templates and get inconsistent data each time.

Hard to feel good when you can't trust your data.

Update

This doesn't happen on the master index. All queries on master are consistent. Even with the log errors.

Here is the crawl log

7504 19:17:39 WARN  IndexCustodian. FullRebuild triggered on index sitecore_web_index.
8068 19:17:41 INFO  [Index=sitecore_web_index] Crawler: Processed 1000 items
9760 19:17:43 INFO  [Index=sitecore_web_index] Crawler: Processed 2000 items
 888 19:17:45 INFO  [Index=sitecore_web_index] Crawler: Processed 3000 items
 888 19:17:46 INFO  [Index=sitecore_web_index] Crawler: Processed 4000 items
4592 19:17:48 INFO  [Index=sitecore_web_index] Crawler: Processed 5000 items
 888 19:17:50 INFO  [Index=sitecore_web_index] Crawler: Processed 6000 items
 888 19:17:52 INFO  [Index=sitecore_web_index] Crawler: Processed 7000 items
4592 19:17:53 INFO  [Index=sitecore_web_index] Crawler: Processed 8000 items
8068 19:17:55 INFO  [Index=sitecore_web_index] Crawler: Processed 9000 items
9360 19:17:57 INFO  [Index=sitecore_web_index] Crawler: Processed 10000 items
4592 19:17:57 WARN  [Index=sitecore_web_index] Crawler : AddRecursive DoItemAdd failed - {D88BA5D2-C867-41DD-A9C3-75463F0D82CD}
Exception: System.ArgumentException
Message: '', hexadecimal value 0x01, is an invalid character.
Source: System.Xml
   at System.Xml.XmlEncodedRawTextWriter.WriteElementTextBlock(Char* pSrc, Char* pSrcEnd)
   at System.Xml.XmlEncodedRawTextWriter.WriteString(String text)
   at System.Xml.XmlWellFormedWriter.WriteString(String text)
   at System.Xml.Linq.ElementWriter.WriteElement(XElement e)
   at System.Xml.Linq.XElement.WriteTo(XmlWriter writer)
   at System.Xml.Linq.XNode.GetXmlString(SaveOptions o)
   at SolrNet.Commands.AddCommand`1.ConvertToXml()
   at SolrNet.Commands.AddCommand`1.Execute(ISolrConnection connection)
   at SolrNet.Impl.LowLevelSolrServer.SendAndParseHeader(ISolrCommand cmd)
   at Sitecore.ContentSearch.SolrProvider.SolrBatchUpdateContext.AddRange(IEnumerable`1 group, Int32 groupSize)
   at Sitecore.ContentSearch.SolrProvider.SolrBatchUpdateContext.AddDocument(Object itemToAdd, IExecutionContext[] executionContexts)
   at Sitecore.ContentSearch.SolrProvider.SolrIndexOperations.ApplyPermissionsThenIndex(IProviderUpdateContext context, IIndexable version)
   at Sitecore.ContentSearch.SitecoreItemCrawler.DoAdd(IProviderUpdateContext context, SitecoreIndexableItem indexable)
   at Sitecore.ContentSearch.HierarchicalDataCrawler`1.CrawlItem(T indexable, IProviderUpdateContext context, CrawlState`1 state)

8068 19:17:58 WARN  [Index=sitecore_web_index] Crawler : AddRecursive DoItemAdd failed - {1867C2A4-9A05-44B7-8EE2-0E8F1F3FA1F8}
Exception: System.ArgumentException
Message: '', hexadecimal value 0x01, is an invalid character.
Source: System.Xml
   at System.Xml.XmlEncodedRawTextWriter.WriteElementTextBlock(Char* pSrc, Char* pSrcEnd)
   at System.Xml.XmlEncodedRawTextWriter.WriteString(String text)
   at System.Xml.XmlWellFormedWriter.WriteString(String text)
   at System.Xml.Linq.ElementWriter.WriteElement(XElement e)
   at System.Xml.Linq.XElement.WriteTo(XmlWriter writer)
   at System.Xml.Linq.XNode.GetXmlString(SaveOptions o)
   at SolrNet.Commands.AddCommand`1.ConvertToXml()
   at SolrNet.Commands.AddCommand`1.Execute(ISolrConnection connection)
   at SolrNet.Impl.LowLevelSolrServer.SendAndParseHeader(ISolrCommand cmd)
   at Sitecore.ContentSearch.SolrProvider.SolrBatchUpdateContext.AddRange(IEnumerable`1 group, Int32 groupSize)
   at Sitecore.ContentSearch.SolrProvider.SolrBatchUpdateContext.AddDocument(Object itemToAdd, IExecutionContext[] executionContexts)
   at Sitecore.ContentSearch.SolrProvider.SolrIndexOperations.ApplyPermissionsThenIndex(IProviderUpdateContext context, IIndexable version)
   at Sitecore.ContentSearch.SitecoreItemCrawler.DoAdd(IProviderUpdateContext context, SitecoreIndexableItem indexable)
   at Sitecore.ContentSearch.HierarchicalDataCrawler`1.CrawlItem(T indexable, IProviderUpdateContext context, CrawlState`1 state)

 888 19:17:58 INFO  [Index=sitecore_web_index] Crawler: Processed 11000 items
9360 19:18:00 WARN  [Index=sitecore_web_index] Crawler : AddRecursive DoItemAdd failed - {5C299C97-BD08-4361-ACDB-538D0AD7491D}
Exception: System.ArgumentException
Message: '.', hexadecimal value 0x00, is an invalid character.
Source: System.Xml
   at System.Xml.XmlEncodedRawTextWriter.WriteElementTextBlock(Char* pSrc, Char* pSrcEnd)
   at System.Xml.XmlEncodedRawTextWriter.WriteString(String text)
   at System.Xml.XmlWellFormedWriter.WriteString(String text)
   at System.Xml.Linq.ElementWriter.WriteElement(XElement e)
   at System.Xml.Linq.XElement.WriteTo(XmlWriter writer)
   at System.Xml.Linq.XNode.GetXmlString(SaveOptions o)
   at SolrNet.Commands.AddCommand`1.ConvertToXml()
   at SolrNet.Commands.AddCommand`1.Execute(ISolrConnection connection)
   at SolrNet.Impl.LowLevelSolrServer.SendAndParseHeader(ISolrCommand cmd)
   at Sitecore.ContentSearch.SolrProvider.SolrBatchUpdateContext.AddRange(IEnumerable`1 group, Int32 groupSize)
   at Sitecore.ContentSearch.SolrProvider.SolrBatchUpdateContext.AddDocument(Object itemToAdd, IExecutionContext[] executionContexts)
   at Sitecore.ContentSearch.SolrProvider.SolrIndexOperations.ApplyPermissionsThenIndex(IProviderUpdateContext context, IIndexable version)
   at Sitecore.ContentSearch.SitecoreItemCrawler.DoAdd(IProviderUpdateContext context, SitecoreIndexableItem indexable)
   at Sitecore.ContentSearch.HierarchicalDataCrawler`1.CrawlItem(T indexable, IProviderUpdateContext context, CrawlState`1 state)

 888 19:18:10 INFO  [Index=sitecore_web_index] Crawler: Processed 12000 items
8068 19:18:22 WARN  [Index=sitecore_web_index] Crawler : AddRecursive DoItemAdd failed - {527BFD42-D34E-436C-9683-C00A6ED1E355}
Exception: System.ArgumentException
Message: '.', hexadecimal value 0x00, is an invalid character.
Source: System.Xml
   at System.Xml.XmlEncodedRawTextWriter.WriteElementTextBlock(Char* pSrc, Char* pSrcEnd)
   at System.Xml.XmlEncodedRawTextWriter.WriteString(String text)
   at System.Xml.XmlWellFormedWriter.WriteString(String text)
   at System.Xml.Linq.ElementWriter.WriteElement(XElement e)
   at System.Xml.Linq.XElement.WriteTo(XmlWriter writer)
   at System.Xml.Linq.XNode.GetXmlString(SaveOptions o)
   at SolrNet.Commands.AddCommand`1.ConvertToXml()
   at SolrNet.Commands.AddCommand`1.Execute(ISolrConnection connection)
   at SolrNet.Impl.LowLevelSolrServer.SendAndParseHeader(ISolrCommand cmd)
   at Sitecore.ContentSearch.SolrProvider.SolrBatchUpdateContext.AddRange(IEnumerable`1 group, Int32 groupSize)
   at Sitecore.ContentSearch.SolrProvider.SolrBatchUpdateContext.AddDocument(Object itemToAdd, IExecutionContext[] executionContexts)
   at Sitecore.ContentSearch.SolrProvider.SolrIndexOperations.ApplyPermissionsThenIndex(IProviderUpdateContext context, IIndexable version)
   at Sitecore.ContentSearch.SitecoreItemCrawler.DoAdd(IProviderUpdateContext context, SitecoreIndexableItem indexable)
   at Sitecore.ContentSearch.HierarchicalDataCrawler`1.CrawlItem(T indexable, IProviderUpdateContext context, CrawlState`1 state)

9360 19:18:23 INFO  [Index=sitecore_web_index] Crawler: Processed 13000 items
9360 19:18:25 INFO  [Index=sitecore_web_index] Crawler: Processed 14000 items
8068 19:18:27 INFO  [Index=sitecore_web_index] Crawler: Processed 15000 items
4592 19:18:29 INFO  [Index=sitecore_web_index] Crawler: Processed 16000 items
9360 19:18:30 INFO  [Index=sitecore_web_index] Crawler: Processed 17000 items
9760 19:18:32 INFO  [Index=sitecore_web_index] Crawler: Processed 18000 items
8068 19:18:34 INFO  [Index=sitecore_web_index] Crawler: Processed 19000 items
9760 19:18:35 INFO  [Index=sitecore_web_index] Crawler: Processed 20000 items
9360 19:18:37 INFO  [Index=sitecore_web_index] Crawler: Processed 21000 items
9360 19:18:39 INFO  [Index=sitecore_web_index] Crawler: Processed 22000 items
4592 19:18:41 INFO  [Index=sitecore_web_index] Crawler: Processed 23000 items
9760 19:18:42 INFO  [Index=sitecore_web_index] Crawler: Processed 24000 items
4592 19:18:52 INFO  [Index=sitecore_web_index] Crawler: Processed 25000 items
 888 19:18:54 INFO  [Index=sitecore_web_index] Crawler: Processed 26000 items
 888 19:18:57 INFO  [Index=sitecore_web_index] Crawler: Processed 27000 items
9760 19:18:59 INFO  [Index=sitecore_web_index] Crawler: Processed 28000 items
9760 19:19:01 INFO  [Index=sitecore_web_index] Crawler: Processed 29000 items
9760 19:19:03 INFO  [Index=sitecore_web_index] Crawler: Processed 30000 items
8068 19:19:05 INFO  [Index=sitecore_web_index] Crawler: Processed 31000 items
9360 19:19:07 INFO  [Index=sitecore_web_index] Crawler: Processed 32000 items
8068 19:19:09 INFO  [Index=sitecore_web_index] Crawler: Processed 33000 items
 888 19:19:12 INFO  [Index=sitecore_web_index] Crawler: Processed 34000 items
4592 19:19:14 INFO  [Index=sitecore_web_index] Crawler: Processed 35000 items
4592 19:19:16 INFO  [Index=sitecore_web_index] Crawler: Processed 36000 items
9760 19:19:19 INFO  [Index=sitecore_web_index] Crawler: Processed 37000 items
9360 19:19:21 INFO  [Index=sitecore_web_index] Crawler: Processed 38000 items
 888 19:19:23 INFO  [Index=sitecore_web_index] Crawler: Processed 39000 items
4592 19:19:25 INFO  [Index=sitecore_web_index] Crawler: Processed 40000 items
 888 19:19:27 INFO  [Index=sitecore_web_index] Crawler: Processed 41000 items
4592 19:19:29 INFO  [Index=sitecore_web_index] Crawler: Processed 42000 items
 888 19:19:32 INFO  [Index=sitecore_web_index] Crawler: Processed 43000 items
 888 19:19:34 INFO  [Index=sitecore_web_index] Crawler: Processed 44000 items
4592 19:19:36 INFO  [Index=sitecore_web_index] Crawler: Processed 45000 items
9360 19:19:38 INFO  [Index=sitecore_web_index] Crawler: Processed 46000 items
9760 19:19:40 INFO  [Index=sitecore_web_index] Crawler: Processed 47000 items
 888 19:19:42 INFO  [Index=sitecore_web_index] Crawler: Processed 48000 items
9760 19:19:44 INFO  [Index=sitecore_web_index] Crawler: Processed 49000 items
 888 19:19:47 INFO  [Index=sitecore_web_index] Crawler: Processed 50000 items
8068 19:19:49 INFO  [Index=sitecore_web_index] Crawler: Processed 51000 items
 888 19:19:51 INFO  [Index=sitecore_web_index] Crawler: Processed 52000 items
 888 19:19:52 INFO  [Index=sitecore_web_index] Crawler: Processed 53000 items
8068 19:19:54 INFO  [Index=sitecore_web_index] Crawler: Processed 54000 items
8068 19:19:56 INFO  [Index=sitecore_web_index] Crawler: Processed 55000 items
8068 19:19:57 INFO  [Index=sitecore_web_index] Crawler: Processed 56000 items
4592 19:19:59 INFO  [Index=sitecore_web_index] Crawler: Processed 57000 items
 888 19:20:00 INFO  [Index=sitecore_web_index] Crawler: Processed 58000 items
 888 19:20:02 INFO  [Index=sitecore_web_index] Crawler: Processed 59000 items
4592 19:20:05 INFO  [Index=sitecore_web_index] Crawler: Processed 60000 items
9760 19:20:07 INFO  [Index=sitecore_web_index] Crawler: Processed 61000 items
4592 19:20:08 INFO  [Index=sitecore_web_index] Crawler: Processed 62000 items
8068 19:20:10 INFO  [Index=sitecore_web_index] Crawler: Processed 63000 items
4592 19:20:12 INFO  [Index=sitecore_web_index] Crawler: Processed 64000 items
8068 19:20:14 INFO  [Index=sitecore_web_index] Crawler: Processed 65000 items
8068 19:20:16 INFO  [Index=sitecore_web_index] Crawler: Processed 66000 items
8068 19:20:18 INFO  [Index=sitecore_web_index] Crawler: Processed 67000 items
4592 19:20:20 INFO  [Index=sitecore_web_index] Crawler: Processed 68000 items
8068 19:20:22 INFO  [Index=sitecore_web_index] Crawler: Processed 69000 items
9760 19:20:24 INFO  [Index=sitecore_web_index] Crawler: Processed 70000 items
 888 19:20:26 INFO  [Index=sitecore_web_index] Crawler: Processed 71000 items
 888 19:20:28 INFO  [Index=sitecore_web_index] Crawler: Processed 72000 items
8068 19:20:31 INFO  [Index=sitecore_web_index] Crawler: Processed 73000 items
4592 19:20:33 INFO  [Index=sitecore_web_index] Crawler: Processed 74000 items
 888 19:20:35 INFO  [Index=sitecore_web_index] Crawler: Processed 75000 items
9760 19:20:37 INFO  [Index=sitecore_web_index] Crawler: Processed 76000 items
9760 19:20:39 INFO  [Index=sitecore_web_index] Crawler: Processed 77000 items
4592 19:20:42 INFO  [Index=sitecore_web_index] Crawler: Processed 78000 items
4592 19:20:43 INFO  [Index=sitecore_web_index] Crawler: Processed 79000 items
8068 19:20:45 INFO  [Index=sitecore_web_index] Crawler: Processed 80000 items
9760 19:20:47 INFO  [Index=sitecore_web_index] Crawler: Processed 81000 items
9360 19:20:49 INFO  [Index=sitecore_web_index] Crawler: Processed 82000 items
9360 19:20:51 INFO  [Index=sitecore_web_index] Crawler: Processed 83000 items
9760 19:20:53 INFO  [Index=sitecore_web_index] Crawler: Processed 84000 items
4592 19:20:55 INFO  [Index=sitecore_web_index] Crawler: Processed 85000 items
8068 19:20:57 INFO  [Index=sitecore_web_index] Crawler: Processed 86000 items
 888 19:20:59 INFO  [Index=sitecore_web_index] Crawler: Processed 87000 items
9360 19:21:02 INFO  [Index=sitecore_web_index] Crawler: Processed 88000 items
4592 19:21:06 INFO  [Index=sitecore_web_index] Crawler: Processed 89000 items
8068 19:21:11 INFO  [Index=sitecore_web_index] Crawler: Processed 90000 items
 888 19:21:14 INFO  [Index=sitecore_web_index] Crawler: Processed 91000 items
8068 19:21:18 INFO  [Index=sitecore_web_index] Crawler: Processed 92000 items
4592 19:21:22 INFO  [Index=sitecore_web_index] Crawler: Processed 93000 items
9760 19:21:26 INFO  [Index=sitecore_web_index] Crawler: Processed 94000 items
8068 19:21:28 INFO  [Index=sitecore_web_index] Crawler: Processed 95000 items
 888 19:21:30 INFO  [Index=sitecore_web_index] Crawler: Processed 96000 items
 888 19:21:32 INFO  [Index=sitecore_web_index] Crawler: Processed 97000 items
8068 19:21:34 INFO  [Index=sitecore_web_index] Crawler: Processed 98000 items
8068 19:21:36 INFO  [Index=sitecore_web_index] Crawler: Processed 99000 items
4592 19:21:38 INFO  [Index=sitecore_web_index] Crawler: Processed 100000 items
8068 19:21:40 INFO  [Index=sitecore_web_index] Crawler: Processed 101000 items
 888 19:21:42 INFO  [Index=sitecore_web_index] Crawler: Processed 102000 items
8068 19:21:44 INFO  [Index=sitecore_web_index] Crawler: Processed 103000 items
8068 19:21:46 INFO  [Index=sitecore_web_index] Crawler: Processed 104000 items
8068 19:21:49 INFO  [Index=sitecore_web_index] Crawler: Processed 105000 items
 888 19:21:51 INFO  [Index=sitecore_web_index] Crawler: Processed 106000 items
9760 19:21:54 INFO  [Index=sitecore_web_index] Crawler: Processed 107000 items
9360 19:21:56 WARN  [Index=sitecore_web_index] Crawler : AddRecursive DoItemAdd failed - {67418AA9-57AC-42E8-A6D1-3D75C01DA680}
Exception: System.ArgumentException
Message: '', hexadecimal value 0x12, is an invalid character.
Source: System.Xml
   at System.Xml.XmlEncodedRawTextWriter.WriteElementTextBlock(Char* pSrc, Char* pSrcEnd)
   at System.Xml.XmlEncodedRawTextWriter.WriteString(String text)
   at System.Xml.XmlWellFormedWriter.WriteString(String text)
   at System.Xml.Linq.ElementWriter.WriteElement(XElement e)
   at System.Xml.Linq.XElement.WriteTo(XmlWriter writer)
   at System.Xml.Linq.XNode.GetXmlString(SaveOptions o)
   at SolrNet.Commands.AddCommand`1.ConvertToXml()
   at SolrNet.Commands.AddCommand`1.Execute(ISolrConnection connection)
   at SolrNet.Impl.LowLevelSolrServer.SendAndParseHeader(ISolrCommand cmd)
   at Sitecore.ContentSearch.SolrProvider.SolrBatchUpdateContext.AddRange(IEnumerable`1 group, Int32 groupSize)
   at Sitecore.ContentSearch.SolrProvider.SolrBatchUpdateContext.AddDocument(Object itemToAdd, IExecutionContext[] executionContexts)
   at Sitecore.ContentSearch.SolrProvider.SolrIndexOperations.ApplyPermissionsThenIndex(IProviderUpdateContext context, IIndexable version)
   at Sitecore.ContentSearch.SitecoreItemCrawler.DoAdd(IProviderUpdateContext context, SitecoreIndexableItem indexable)
   at Sitecore.ContentSearch.HierarchicalDataCrawler`1.CrawlItem(T indexable, IProviderUpdateContext context, CrawlState`1 state)

9760 19:21:57 INFO  [Index=sitecore_web_index] Crawler: Processed 108000 items
4592 19:21:59 INFO  [Index=sitecore_web_index] Crawler: Processed 109000 items
 888 19:22:02 INFO  [Index=sitecore_web_index] Crawler: Processed 110000 items
9360 19:22:04 INFO  [Index=sitecore_web_index] Crawler: Processed 111000 items
 888 19:22:06 INFO  [Index=sitecore_web_index] Crawler: Processed 112000 items
8068 19:22:10 INFO  [Index=sitecore_web_index] Crawler: Processed 113000 items
9760 19:22:13 INFO  [Index=sitecore_web_index] Crawler: Processed 114000 items
8068 19:22:16 INFO  [Index=sitecore_web_index] Crawler: Processed 115000 items
9360 19:22:18 INFO  [Index=sitecore_web_index] Crawler: Processed 116000 items
9360 19:22:20 INFO  [Index=sitecore_web_index] Crawler: Processed 117000 items
 888 19:22:22 INFO  [Index=sitecore_web_index] Crawler: Processed 118000 items
9760 19:22:24 INFO  [Index=sitecore_web_index] Crawler: Processed 119000 items
9760 19:22:26 INFO  [Index=sitecore_web_index] Crawler: Processed 120000 items
4592 19:22:29 INFO  [Index=sitecore_web_index] Crawler: Processed 121000 items
8068 19:22:31 INFO  [Index=sitecore_web_index] Crawler: Processed 122000 items
8068 19:22:33 INFO  [Index=sitecore_web_index] Crawler: Processed 123000 items
9360 19:22:36 INFO  [Index=sitecore_web_index] Crawler: Processed 124000 items
9360 19:22:38 INFO  [Index=sitecore_web_index] Crawler: Processed 125000 items
9360 19:22:40 INFO  [Index=sitecore_web_index] Crawler: Processed 126000 items
4592 19:22:43 INFO  [Index=sitecore_web_index] Crawler: Processed 127000 items
ManagedPoolThread #7 19:22:44 INFO  [Index=sitecore_web_index] Crawler: Total processed items 127587

Update 2

The first thing Support wants me to do is look for bad characters in my content to cut down on the errors. They have given me this SQL to identify the fields and items where bad content is located.

SELECT * FROM (
    SELECT ItemId, FieldId, Value FROM [usms_master].[dbo].[SharedFields] 
    UNION
    SELECT ItemId, FieldId, Value FROM [usms_master].[dbo].[UnversionedFields]
    UNION
    SELECT ItemId, FieldId, Value FROM [usms_master].[dbo].[VersionedFields]
  ) A
WHERE Value Like '%' + CHAR(0x01) + '%'
OR Value Like '%' + CHAR(0x08) + '%'
OR Value Like '%' + CHAR(0x10) + '%'
OR Value Like '%' + CHAR(0x12) + '%'

Second thing is to disable searching of PDFs and docs. They want me to remove these two sections and reindex.

<mediaIndexing hint="skip">
            <mimeTypes>
               <includes>
                  <mimeType>application/pdf</mimeType> <--- remove this
            <extensions>
               ...
               <includes>
                 ...
                 <extension>doc</extension> <--- remove this
  • From my experience, it is those indexing warnings / errors that make the difference. They need to be fixed otherwise sitecore will just skip those items. Why is it random though? Not sure, could be a bug in sitecore / solr. Please post the logs and Solr version. – Mark Gibbons May 22 at 1:01
  • Thanks @MarkGibbons I added the version, 7.2.1 and the SOLR log for the crawl. – Chris Auer May 22 at 1:15
  • Thanks Chris. Are the items that it's complaining about PDF items / media items by any chance? You might need to install the filter doc.sitecore.com/developers/91/… – Mark Gibbons May 22 at 5:18
  • Are you importing PDFs from Sharepoint by chance? There is some XML in the metadata fields that can cause errors with your indexing. – Goat83 Jun 7 at 20:14
  • This issue is in the encoding. Somehow solr can not recognize that text is in Unicode while written here: at System.Xml.XmlEncodedRawTextWriter.WriteString(String text) – x3mxray Jun 7 at 20:46
4

The issue had a group of issues and all were around errors. Some you can see, some that are not in logs.

As stated in my OP updates, my items has some illegal characters. I needed to run this SQL to find the items and the fields. Then you have to manually find the character and remove it.

SELECT * FROM (
    SELECT ItemId, FieldId, Value FROM [usms_master].[dbo].[SharedFields] 
    UNION
    SELECT ItemId, FieldId, Value FROM [usms_master].[dbo].[UnversionedFields]
    UNION
    SELECT ItemId, FieldId, Value FROM [usms_master].[dbo].[VersionedFields]
  ) A
WHERE Value Like '%' + CHAR(0x01) + '%'
OR Value Like '%' + CHAR(0x08) + '%'
OR Value Like '%' + CHAR(0x10) + '%'
OR Value Like '%' + CHAR(0x12) + '%'

Seconds issue is that Sitecore has a bug with indexing pdfs and word docs. I had to rem out the pdf and doc inclusions in the solr config file.

App_Config\Sitecore\ContentSearch\Sitecore.ContentSearch.Solr.DefaultIndexConfiguration.config

<mediaIndexing hint="skip">
            <mimeTypes>
               <includes>
                  <mimeType>application/pdf</mimeType> <--- remove this
            <extensions>
               ...
               <includes>
                 ...
                 <extension>doc</extension> <--- remove this

There is an available hotfix for bug #305015 for PDF. But there is no hotfix available for the doc issue.

  • I have the same error on SC9.1 and Solr 7.2.1. Executed SQL query but no records found with illegal name, disabled pdf and doc indexing as mentioned above but the same issue persists while rebuilding the master index. Was any hotfix provided by Sitecore for this issue? – Pankaj Tiwari Aug 27 at 22:18
  • Did you do the doc and pdf config changes? The illegal characters are only a small part of the issue. – Chris Auer Aug 27 at 22:22
  • Yes, as mentioned above, changes to disable pdf, doc indexing is implemented as you suggested – Pankaj Tiwari Aug 27 at 22:30
  • Even its same error message while building another index: 00:41:00 WARN [Index=sitecore_marketing_asset_index_master] Crawler : AddRecursive DoItemAdd failed - {27462B5D-0CFC-4001-BA4F-6FD607356C72} Exception: System.ArgumentException Message: '.', hexadecimal value 0x00, is an invalid character. Source: System.Xml at System.Xml.XmlEncodedRawTextWriter.WriteElementTextBlock(Char* pSrc, Char* pSrcEnd) at System.Xml.XmlEncodedRawTextWriter.WriteString(String text) at System.Xml.XmlWellFormedWriter.WriteString(String text) – Pankaj Tiwari Aug 27 at 22:43
  • 1
    The sitecore ticket is 532499 – Chris Auer Aug 27 at 23:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.