How to deploy Solr new custom cores in an AKS environment via Azure DevOps

Question

Regarding the following article: https://www.koenheye.be/running-a-container-to-check-its-contents/ it is possible to create custom cores in the Solr docker container by configuring a specific json file. This happens at T0, through the build of the solr-init image which executes a PowerShell script (Start.ps1) that takes care of creating the cores specified in the JSON file. In order to have Continuous Integration (CI) and Continuous Deployment (CD) processes where it is possible to add other cores, a new build of solr-init would fail, because the JSON file contains cores already present in Solr. What would be the best approach to add new Solr cores via DevOps?

Answer 1

After copying Start.ps1 from solr-init locally, I changed it as follows:

param(
    [Parameter(Mandatory)]
    [string]$SitecoreSolrConnectionString,

    [Parameter(Mandatory)]
    [string]$SolrSitecoreConfigsetSuffixName,

    [Parameter(Mandatory)]
    [string]$SolrCorePrefix,

    [Parameter(Mandatory)]
    [string]$SolrReplicationFactor,

    [Parameter(Mandatory)]
    [int]$SolrNumberOfShards,

    [Parameter(Mandatory)]
    [int]$SolrMaxShardsPerNodes,

    [string]$SolrXdbSchemaFile,

    [string]$SolrCollectionsToDeploy
)

function GetCoreNames {
    param (
        [ValidateSet("sitecore", "xdb")]
        [string]$CoreType, 
        [string]$SolrCollectionsToDeploy,
        [string] $SolrCorePrefix,
        [string[]]$solrCollections
    )

    $resultCoreNames = @()
    $SolrCollectionsToDeploy.Split(',') | ForEach-Object {
        $solrCollectionToDeploy = $_
        Get-ChildItem C:\data -Filter "cores*$solrCollectionToDeploy.json" |         ForEach-Object {
            $coreNames = (Get-Content $_.FullName | Out-String | ConvertFrom-Json).$CoreType
        
            if ($coreNames) {
                foreach ($solrCoreName in $coreNames) {
                    if (($solrCollections -notcontains ('{0}{1}' -f $SolrCorePrefix, $solrCoreName))) {
                    $resultCoreNames += $solrCoreName
                    }    
                }
            }
        }
    }

    return $resultCoreNames
}

. .\Get-SolrCredential.ps1

$solrContext = .\Parse-ConnectionString.ps1 -SitecoreSolrConnectionString         $SitecoreSolrConnectionString

$SolrEndpoint = $solrContext.SolrEndpoint
$env:SOLR_USERNAME = $solrContext.SolrUsername
$env:SOLR_PASSWORD = $solrContext.SolrPassword

$solrCollections = (Invoke-RestMethod -Uri "$SolrEndpoint/admin/collections?    action=LIST&omitHeader=true" -Method Get -Credential (Get-    SolrCredential)).collections
$solrSitecoreCoreNames = GetCoreNames -CoreType "sitecore" -SolrCollectionsToDeploy $SolrCollectionsToDeploy $SolrCorePrefix $solrCollections
$solrXdbCoreNames = GetCoreNames -CoreType "xdb" -SolrCollectionsToDeploy     $SolrCollectionsToDeploy $SolrCorePrefix $solrCollections


$solrConfigDir = "C:\temp\sitecore_content_config"
$solrBaseConfigDir = "C:\temp\default"
.\Download-SolrConfig.ps1 -SolrEndpoint $SolrEndpoint -OutPath $solrBaseConfigDir
.\Patch-SolrConfig.ps1 -SolrConfigPath $solrBaseConfigDir -XsltPath     "C:\data\xslt" -OutputPath $solrConfigDir

$collectionAliases = $null
if(Test-Path -Path "C:\data\aliases.json") {
    $collectionAliases = ((Get-Content C:\data\aliases.json | Out-String |     ConvertFrom-Json).aliases)
}

if ($solrSitecoreCoreNames)
{
    $solrBaseConfigsetName = $null
    foreach ($solrCoreName in $solrSitecoreCoreNames) {
        $solrConfigsetName = ('{0}{1}{2}' -f $SolrCorePrefix, $solrCoreName,     $SolrSitecoreConfigsetSuffixName)

        if ([string]::IsNullOrEmpty($solrBaseConfigsetName)) {
            .\New-SolrConfig.ps1 -SolrEndpoint $SolrEndpoint -SolrConfigName     $solrConfigsetName -SolrConfigDir $solrConfigDir
            $solrBaseConfigsetName = $solrConfigsetName
        }
        else {
            .\Copy-SolrConfig.ps1 -SolrEndpoint $SolrEndpoint -SolrConfigName     $solrConfigsetName -SolrBaseConfigName $solrBaseConfigsetName
        }

        .\New-SolrCore.ps1 -SolrCoreNames $solrCoreName -SolrEndpoint     $SolrEndpoint -SolrCorePrefix $SolrCorePrefix -SolrConfigsetName $solrConfigsetName     -SolrReplicationFactor $SolrReplicationFactor -SolrNumberOfShards     $SolrNumberOfShards -SolrMaxShardNumberPerNode $SolrMaxShardsPerNodes
    }
}

if($solrXdbCoreNames) 
{
    foreach ($solrCoreName in $solrXdbCoreNames) {
        $SolrXdbConfigsetName = ('{0}{1}{2}' -f $SolrCorePrefix, $solrCoreName,     $SolrSitecoreConfigsetSuffixName)
        $SolrCollectionName = ('{0}{1}' -f $SolrCorePrefix, $solrCoreName)

        if($solrconfigs -notcontains $SolrXdbConfigsetName)
        {
             .\Copy-SolrConfig.ps1 -SolrEndpoint $SolrEndpoint -SolrConfigName     $SolrXdbConfigsetName
        }
        .\New-SolrCore.ps1 -SolrCoreNames $solrCoreName -SolrEndpoint         $SolrEndpoint -SolrCorePrefix $SolrCorePrefix -SolrConfigsetName         $SolrXdbConfigsetName -SolrReplicationFactor $SolrReplicationFactor -        SolrNumberOfShards $SolrNumberOfShards -SolrMaxShardNumberPerNode         $SolrMaxShardsPerNodes -SolrCollectionAliases $collectionAliases
    
        .\Update-Schema.ps1 -SolrCollectionName $SolrCollectionName -SolrEndpoint $SolrEndpoint -SchemaPath $SolrXdbSchemaFile
     }
 }

In this way, the PowerShell script could be used to create only the cores not present in Solr. To apply the change and replace the old Start.ps1 file, I had to change the DockerFile related to the solr-init build as follows:

# escape = `

IMAGE_BASE ARG

FROM $ {BASE_IMAGE}

SHELL ["powershell", "-Command", "$ ErrorActionPreference = 'Stop'; $     ProgressPreference = 'Continue silently';"]

COPY Start.ps1 C:\Start.ps1

COPY my-new-cores.json C:\data\my-new-cores.json

At this point it is sufficient to execute the build of solr-init image in the CI pipeline and that's it