1

We are using Sitecore 10.1.3 on Managed Cloud.

The website has a Sitecore form with two Submit Actions - Call_API & Show_Success_Page.

The Call_API action calls our applications custom back-end method Execute, which has line-by-line logging right from its first line and it includes the data being sent to the API and also the response received from API.

Based on the external API database data, we have realized that only some form submissions are captured there.

Irrespective of the submission success/failure, the visitor would still see the "Success Page", as this second submit action is a redirection without any validation.

On checking the logs in Azure App Insights, the custom logs are available only for some of the submissions.

When a visitor, submits a form, in the browser's network tab, we can see that the first request will be for the default formbuilder?...., which is logged by Sitecore also.

In the logs, for the missing submissions, even formbuilder?.... log is unavailable. For the successful submissions all the custom logs are being logged.

Looks like the Execute method itself is not being hit!

Seems like, somehow on the form's submit button hit, it is assuming a success (or return true) and then proceeding to the next submit action without even hitting the back-end code.

  1. Is there any firewall bot detection at the Azure side for Sitecore Managed Cloud platforms? If yes, can we disable it?
  2. To add to my confusion, there are two "bot detection" features in Sitecore:

One is in the Submit Action item - Enable Robot Submission - Select to allow bot form submission and another, for the Form item - Is Robot Detection Enabled.

I have disabled Is Robot Detection Enabled and also cleared cache at /sitecore/admin/cache.aspx.

Tried some submissions and all were successful, but I'm not sure which of the fixes fixed this.

Could you please help me understand, if any of those "bot detection" features could be responsible for this issue.

Both features look similar. When to use which one or one over other or both?

The concern is, there could be valid submissions which did not go through because a bot detection was enabled.

Is there a way such bot submissions are logged anywhere in Sitecore?

Any way I can troubleshoot or isolate the issue?

Thank you

1 Answer 1

2

I've seen a very similar problem with forms data not being saved occasionally. It was caused by Sitecore Forms bot detection functionality blocking some of the submit actions. In most cases the issue is somewhere in Sitecore tracking and visitor identification, because in order to classify a visitor as a bot or human, Sitecore needs to have access to the current session tracker, user's contact ID and be able to run visitor identification script.

Even though disabling bot detection can be a quick workaround, I would recommend investigating why real users are mistakenly blocked by bot detection.

To answer your questions:

Is there any firewall bot detection at the Azure side for Sitecore Managed Cloud platforms? If yes, can we disable it?

An App Gateway or CDN with web application firewall (WAF) can be configured in front of Sitecore infrastructure, but if a formbuilder.aspx request was blocked by the WAF, you would not even see it in the app service logs.

To add to my confusion, there are two "bot detection" features in Sitecore:

Yes, bot detection for forms can be enabled/disabled on two levels:

  • for the entire form by setting the Robot detection enabled checkbox on the form level
  • for each submit action separately by setting the Enable Robot Submission checkbox on the submit action level

By default bot detection is enabled for all forms and for those submit actions that involve saving data. This is why the redirect to "Thank You" page works fine even for bots - bot detection is usually disabled for this submit action. However, it makes it difficult to spot this problem because in the front-end all form submissions seem to be successful.

Tried some submissions and all were successful, but I'm not sure which of the fixes fixed this.

Either of these checkboxes will help, disabling bot detection on the form level disables it for all submit actions automatically.

Both features look similar. When to use which one or one over other or both?

If you want to disable bot detection for all submit actiona, you can do it on the form level. If you want it to be enabled for the form, but block only specific form actions, for example, sending emails, pushing data to a DB or CRM, then you should use the action-level checkbox.

Is there a way such bot submissions are logged anywhere in Sitecore?

Unfortunately no, by default this form data will not be logged in Sitecore, unless you have some custom code that saves this information.

Any way I can troubleshoot or isolate the issue?

Yes, start from checking the cookie SC_ANALYTICS_GLOBAL_COOKIE. If it's missing, then bot detection will block the form submission. For example, I've seen situations when cookie consent modules or custom pipelines were removing SC_ANALYTICS_GLOBAL_COOKIE and it resulted in lost form submissions. Another potential reason is missing tag @Html.Sitecore().VisitorIdentification() in some of the layouts.

If the cookie exists, look at the second part of it after the | character - it's called IsClassificationGuessed and contains a true or false value. If false, it means that the website has not been able to verify whether the user is bot or not so make sure that the script /layouts/system/VisitorIdentification.js loads and runs as part of the visitor identification tag for new contacts.

Potential season for users being classified as bots:

  • Their user agent is flagged as bot by Device Detection
  • Their IP address is listed in <excludedIPAddresses> config
  • The script /layouts/system/VisitorIdentification.js did not load in the browser or did not detect any mouse movements/touch screen actions

Here are some helpful Sitecore documentation pages with additional details:

2
  • 1
    "If false, it means that the website has not been able to verify whether the user is bot or not". In such case as unknown, will the form be submitted?
    – sukesh
    Nov 3, 2023 at 12:40
  • 1
    @sukesh No, the form will treat such user as a bot. On the very first server request, Sitecore classifies a contact as IsAutoDetectedRobot with classification code "925", this code can later change to "0" for humans or to "930" for known robots. When bot detection is enabled, form submissions are blocked for all contacts with classification code >= 900.
    – Anna Gevel
    Nov 3, 2023 at 13:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.