3

Regarding the following article: https://www.koenheye.be/running-a-container-to-check-its-contents/ it is possible to create custom cores in the Solr docker container by configuring a specific json file. This happens at T0, through the build of the solr-init image which executes a PowerShell script (Start.ps1) that takes care of creating the cores specified in the JSON file. In order to have Continuous Integration (CI) and Continuous Deployment (CD) processes where it is possible to add other cores, a new build of solr-init would fail, because the JSON file contains cores already present in Solr. What would be the best approach to add new Solr cores via DevOps?

5
  • Creating SOLR core would be one time activity. So I believe we should not be having this in CI-CD pipeline. Just to automate this process, we can crate PowerShell script.
    – Jitendra
    May 31 at 5:30
  • Thanks Jitendra, however it is not unlikely that if you need to add new features after the first production release, you will also need to create new indexes (e.g. add a News feature with its custom indexes). If you don't automate this process in some way, after deploying to an AKS environment, the CD and CM referencing those indexes will be unhealty. May 31 at 6:52
  • I was trying to say, we can create index CORE once (a particular index core). With every deployment, we should not be creating index CORE. Otherwise everytime we have to rebuild the index. It will take time and sometime will take days (dependes on size of the index, sitecore). So it should not be part of our CI-CD pipleline.
    – Jitendra
    Jun 1 at 4:25
  • Of course, but I don't want to recreate an index core at each deployment, but add a new one if necessary and if it doesn't exist. In real life it could happen ... and in a containerized environment, with an active health check, the new core must exist before deploying Sitecore roles that reference it Jun 1 at 6:09
  • In this case, I believe you can create a powershell script and call this in CI-CD pipeline.
    – Jitendra
    Jun 1 at 8:55

1 Answer 1

0

After copying Start.ps1 from solr-init locally, I changed it as follows:

param(
    [Parameter(Mandatory)]
    [string]$SitecoreSolrConnectionString,

    [Parameter(Mandatory)]
    [string]$SolrSitecoreConfigsetSuffixName,

    [Parameter(Mandatory)]
    [string]$SolrCorePrefix,

    [Parameter(Mandatory)]
    [string]$SolrReplicationFactor,

    [Parameter(Mandatory)]
    [int]$SolrNumberOfShards,

    [Parameter(Mandatory)]
    [int]$SolrMaxShardsPerNodes,

    [string]$SolrXdbSchemaFile,

    [string]$SolrCollectionsToDeploy
)

function GetCoreNames {
    param (
        [ValidateSet("sitecore", "xdb")]
        [string]$CoreType, 
        [string]$SolrCollectionsToDeploy,
        [string] $SolrCorePrefix,
        [string[]]$solrCollections
    )

    $resultCoreNames = @()
    $SolrCollectionsToDeploy.Split(',') | ForEach-Object {
        $solrCollectionToDeploy = $_
        Get-ChildItem C:\data -Filter "cores*$solrCollectionToDeploy.json" |         ForEach-Object {
            $coreNames = (Get-Content $_.FullName | Out-String | ConvertFrom-Json).$CoreType
        
            if ($coreNames) {
                foreach ($solrCoreName in $coreNames) {
                    if (($solrCollections -notcontains ('{0}{1}' -f $SolrCorePrefix, $solrCoreName))) {
                    $resultCoreNames += $solrCoreName
                    }    
                }
            }
        }
    }

    return $resultCoreNames
}

. .\Get-SolrCredential.ps1

$solrContext = .\Parse-ConnectionString.ps1 -SitecoreSolrConnectionString         $SitecoreSolrConnectionString

$SolrEndpoint = $solrContext.SolrEndpoint
$env:SOLR_USERNAME = $solrContext.SolrUsername
$env:SOLR_PASSWORD = $solrContext.SolrPassword

$solrCollections = (Invoke-RestMethod -Uri "$SolrEndpoint/admin/collections?    action=LIST&omitHeader=true" -Method Get -Credential (Get-    SolrCredential)).collections
$solrSitecoreCoreNames = GetCoreNames -CoreType "sitecore" -SolrCollectionsToDeploy $SolrCollectionsToDeploy $SolrCorePrefix $solrCollections
$solrXdbCoreNames = GetCoreNames -CoreType "xdb" -SolrCollectionsToDeploy     $SolrCollectionsToDeploy $SolrCorePrefix $solrCollections


$solrConfigDir = "C:\temp\sitecore_content_config"
$solrBaseConfigDir = "C:\temp\default"
.\Download-SolrConfig.ps1 -SolrEndpoint $SolrEndpoint -OutPath $solrBaseConfigDir
.\Patch-SolrConfig.ps1 -SolrConfigPath $solrBaseConfigDir -XsltPath     "C:\data\xslt" -OutputPath $solrConfigDir

$collectionAliases = $null
if(Test-Path -Path "C:\data\aliases.json") {
    $collectionAliases = ((Get-Content C:\data\aliases.json | Out-String |     ConvertFrom-Json).aliases)
}

if ($solrSitecoreCoreNames)
{
    $solrBaseConfigsetName = $null
    foreach ($solrCoreName in $solrSitecoreCoreNames) {
        $solrConfigsetName = ('{0}{1}{2}' -f $SolrCorePrefix, $solrCoreName,     $SolrSitecoreConfigsetSuffixName)

        if ([string]::IsNullOrEmpty($solrBaseConfigsetName)) {
            .\New-SolrConfig.ps1 -SolrEndpoint $SolrEndpoint -SolrConfigName     $solrConfigsetName -SolrConfigDir $solrConfigDir
            $solrBaseConfigsetName = $solrConfigsetName
        }
        else {
            .\Copy-SolrConfig.ps1 -SolrEndpoint $SolrEndpoint -SolrConfigName     $solrConfigsetName -SolrBaseConfigName $solrBaseConfigsetName
        }

        .\New-SolrCore.ps1 -SolrCoreNames $solrCoreName -SolrEndpoint     $SolrEndpoint -SolrCorePrefix $SolrCorePrefix -SolrConfigsetName $solrConfigsetName     -SolrReplicationFactor $SolrReplicationFactor -SolrNumberOfShards     $SolrNumberOfShards -SolrMaxShardNumberPerNode $SolrMaxShardsPerNodes
    }
}

if($solrXdbCoreNames) 
{
    foreach ($solrCoreName in $solrXdbCoreNames) {
        $SolrXdbConfigsetName = ('{0}{1}{2}' -f $SolrCorePrefix, $solrCoreName,     $SolrSitecoreConfigsetSuffixName)
        $SolrCollectionName = ('{0}{1}' -f $SolrCorePrefix, $solrCoreName)

        if($solrconfigs -notcontains $SolrXdbConfigsetName)
        {
             .\Copy-SolrConfig.ps1 -SolrEndpoint $SolrEndpoint -SolrConfigName     $SolrXdbConfigsetName
        }
        .\New-SolrCore.ps1 -SolrCoreNames $solrCoreName -SolrEndpoint         $SolrEndpoint -SolrCorePrefix $SolrCorePrefix -SolrConfigsetName         $SolrXdbConfigsetName -SolrReplicationFactor $SolrReplicationFactor -        SolrNumberOfShards $SolrNumberOfShards -SolrMaxShardNumberPerNode         $SolrMaxShardsPerNodes -SolrCollectionAliases $collectionAliases
    
        .\Update-Schema.ps1 -SolrCollectionName $SolrCollectionName -SolrEndpoint $SolrEndpoint -SchemaPath $SolrXdbSchemaFile
     }
 }

In this way, the PowerShell script could be used to create only the cores not present in Solr. To apply the change and replace the old Start.ps1 file, I had to change the DockerFile related to the solr-init build as follows:

# escape = `

IMAGE_BASE ARG

FROM $ {BASE_IMAGE}

SHELL ["powershell", "-Command", "$ ErrorActionPreference = 'Stop'; $     ProgressPreference = 'Continue silently';"]

COPY Start.ps1 C:\Start.ps1

COPY my-new-cores.json C:\data\my-new-cores.json

At this point it is sufficient to execute the build of solr-init image in the CI pipeline and that's it

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.