Alarm clock event subscriber error "Unable to translate bytes"

Question

I'm seen following error on my logs frequently... It seems its related to analytics data submission but couldn't actually find a reason/solution by searching the web

Sitecore Version : 9.1.1

ERROR Exception in alarm clock event subscriber.
Exception: System.Text.DecoderFallbackException
Message: Unable to translate bytes [FF] at index 113 from specified code page to Unicode.
Source: mscorlib
   at System.Text.DecoderExceptionFallbackBuffer.Throw(Byte[] bytesUnknown, Int32 index)
   at System.Text.DecoderExceptionFallbackBuffer.Fallback(Byte[] bytesUnknown, Int32 index)
   at System.Text.DecoderFallbackBuffer.InternalFallback(Byte[] bytes, Byte* pBytes, Char*& chars)
   at System.Text.UTF8Encoding.GetChars(Byte* bytes, Int32 byteCount, Char* chars, Int32 charCount, DecoderNLS baseDecoder)
   at System.Text.DecoderNLS.GetChars(Byte[] bytes, Int32 byteIndex, Int32 byteCount, Char[] chars, Int32 charIndex, Boolean flush)
   at System.Text.DecoderNLS.GetChars(Byte[] bytes, Int32 byteIndex, Int32 byteCount, Char[] chars, Int32 charIndex)
   at System.IO.BinaryReader.ReadString()
   at System.Runtime.Serialization.Formatters.Binary.BinaryConverter.ReadTypeInfo(BinaryTypeEnum binaryTypeEnum, __BinaryParser input, Int32& assemId)
   at System.Runtime.Serialization.Formatters.Binary.BinaryObjectWithMapTyped.Read(__BinaryParser input)
   at System.Runtime.Serialization.Formatters.Binary.__BinaryParser.ReadObjectWithMapTyped(BinaryHeaderEnum binaryHeaderEnum)
   at System.Runtime.Serialization.Formatters.Binary.__BinaryParser.Run()
   at System.Runtime.Serialization.Formatters.Binary.ObjectReader.Deserialize(HeaderHandler handler, __BinaryParser serParser, Boolean fCheck, Boolean isCrossAppDomain, IMethodCallMessage methodCallMessage)
   at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize(Stream serializationStream, HeaderHandler handler, Boolean fCheck, Boolean isCrossAppDomain, IMethodCallMessage methodCallMessage)
   at Sitecore.Analytics.Data.DataAccess.SubmitQueue.FileSubmitQueue.Dequeue()
   at Sitecore.Analytics.SubmitQueueService.WakeUp()
   at Sitecore.Services.AlarmClock.<>c__DisplayClass11_0.<Heartbeat_Beat>b__0(Object state)

Answer 1

Contacted Sitecore Support and its related to corrupted submitQueue folder content. Once cleared the SubmitQueue data, issue went away.

Following is the answer from Sitecore Support:

From the logs you provided it looks like the content of the Submit Queue folder got corrupted.

May I ask you to check if there is any file under \YourDataFolder\Submit Queue ? If there is any file, could you please take a back up copy of them and then delete them from the folder?

After you removed the files please check if you experience the same issue again.