0

I am trying to upload files more than 200 MB in Sitecore Content Hub using Blob Storage URLs. For file size 700 MB it is working fine. But for more than 1 GB files, the Azure Blob SAAS URL is getting timed out. Can anyone help me out with how to increase the timeout for Azure Blob storage URLs from Content Hub or any other way to upload.

I am using the below codes.

byte[] buffer = new byte[size];
using (FileStream fs = File.OpenRead(fileName))
{
    using (BufferedStream bs = new BufferedStream(fs))
    {
        while ((bytesRead = bs.Read(buffer, 0, size)) != 0)
        {
            {
                Console.WriteLine("Block #" + blockNumber);
                blockNumber++;

                string blockIds = $"{blockNumber:0000000}";
                string blockId = Convert.ToBase64String(Encoding.UTF8.GetBytes(blockIds));
                                    
                Console.WriteLine("Bock Id : " + blockId);
                                    
                HttpClient blockClient = new HttpClient();
                blockClient.Timeout = TimeSpan.FromMinutes(30);
                var binaryContent = new ByteArrayContent(buffer);
                binaryContent.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");

                var res1 = await blockClient.PutAsync(uploadLocation + "&comp=block&blockid=" + blockId + "", binaryContent);
                Console.WriteLine(res1.ReasonPhrase);
                blockList.Add(blockId);
                blockListXml += "<Latest>" + blockId + "</Latest>";
                Console.WriteLine("Bytes read : " + bytesRead);
            };
        }
    }

    blockListXml += "</BlockList>";
7
  • Is there a reason that you need to have files at large? Doesn't seem practical for anyone to ever download it. Jul 3, 2021 at 19:54
  • 1
    Can you clean this question up, please? title says you have problems > 200MB but in the post itself it says 700 MB is fine but > 1GB is a problem.
    – Mark Cassidy
    Jul 4, 2021 at 10:11
  • @Mark Sitecore Content Hub allows files up to 200MB, and I have to upload a 5GB file. The blob storage SaaS URL CH is providing is, can be accessed for 5 mins only. Within that 5 mins, I was able to upload 700 MB file only.
    – Arkadeep
    Jul 4, 2021 at 15:40
  • @MichaelWest, it is very common to upload huge files in DAM. Those are created for this purpose I believe.
    – Arkadeep
    Jul 4, 2021 at 15:41
  • @Arkadeep The point I'm trying to make is in what practical way would someone use a 1 GB or even a 5 GB file from the DAM? Videos are likely streamed (such as form YouTube) and full-sized images are generally less than 25 MB. You could be opening a whole can of worms for yourself if you don't get to the bottom of it. Jul 5, 2021 at 3:17

1 Answer 1

0

to address the timeout issues you're experiencing with large files.

Within the Content Hub, timeouts can occur when directly handling sizable assets due to the synchronous nature of some operations. However, to mitigate this, Content Hub provides the FetchJob mechanism. This enables us to fetch content asynchronously from a specified URL, which is especially helpful when managing large files.

    public async Task CreateOrUpdateAsset(long assetId, string videoBlobUrl, bool isAdaptation)
    {
        var existingAsset = await _mClientFactory.Client.Entities.GetAsync(assetId, EntityLoadConfiguration.Full);
        var targetAssetId = assetId;

        // Load the new video asset
        var assetEntity = !isAdaptation ?
            await _mClientFactory.Client.EntityFactory.CreateAsync(Constants.EntityDefinitions.MAsset) :
            existingAsset;

        // Set properties and relations
        if (!isAdaptation)
        {
            var culture = await _mClientFactory.Client.Cultures.GetDefaultCultureAsync();
            await SetAssetProperties(assetEntity, existingAsset, culture);
            await SetAssetRelations(assetEntity, existingAsset);
            targetAssetId = await _mClientFactory.Client.Entities.SaveAsync(assetEntity);
        }
        // await _mClientFactory.Client.Assets.FinalLifeCycleManager.ApproveAsync(assetEntity.Id.Value);

        var jobId = await CreateFetchJobAsync(targetAssetId, videoBlobUrl);

        _logger.LogInformation($"Fetch job created ({jobId}) for {targetAssetId} asset.");
    }

    private async Task<long> CreateFetchJobAsync(long assetId, string url)
    {
        var webFetchJob = new WebFetchJobRequest("Fetches rendered video from the storage.", assetId)
        {
            Urls = new List<Uri> { new Uri(url) }
        };

        return await _mClientFactory.Client.Jobs.CreateFetchJobAsync(webFetchJob);
    }

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.