Upload file more than 200 mb in content hub

Question

I am trying to upload files more than 200 MB in Sitecore Content Hub using Blob Storage URLs. For file size 700 MB it is working fine. But for more than 1 GB files, the Azure Blob SAAS URL is getting timed out. Can anyone help me out with how to increase the timeout for Azure Blob storage URLs from Content Hub or any other way to upload.

I am using the below codes.

byte[] buffer = new byte[size];
using (FileStream fs = File.OpenRead(fileName))
{
    using (BufferedStream bs = new BufferedStream(fs))
    {
        while ((bytesRead = bs.Read(buffer, 0, size)) != 0)
        {
            {
                Console.WriteLine("Block #" + blockNumber);
                blockNumber++;

                string blockIds = $"{blockNumber:0000000}";
                string blockId = Convert.ToBase64String(Encoding.UTF8.GetBytes(blockIds));
                                    
                Console.WriteLine("Bock Id : " + blockId);
                                    
                HttpClient blockClient = new HttpClient();
                blockClient.Timeout = TimeSpan.FromMinutes(30);
                var binaryContent = new ByteArrayContent(buffer);
                binaryContent.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");

                var res1 = await blockClient.PutAsync(uploadLocation + "&comp=block&blockid=" + blockId + "", binaryContent);
                Console.WriteLine(res1.ReasonPhrase);
                blockList.Add(blockId);
                blockListXml += "<Latest>" + blockId + "</Latest>";
                Console.WriteLine("Bytes read : " + bytesRead);
            };
        }
    }

    blockListXml += "</BlockList>";

Answer 1

to address the timeout issues you're experiencing with large files.

Within the Content Hub, timeouts can occur when directly handling sizable assets due to the synchronous nature of some operations. However, to mitigate this, Content Hub provides the FetchJob mechanism. This enables us to fetch content asynchronously from a specified URL, which is especially helpful when managing large files.

    public async Task CreateOrUpdateAsset(long assetId, string videoBlobUrl, bool isAdaptation)
    {
        var existingAsset = await _mClientFactory.Client.Entities.GetAsync(assetId, EntityLoadConfiguration.Full);
        var targetAssetId = assetId;

        // Load the new video asset
        var assetEntity = !isAdaptation ?
            await _mClientFactory.Client.EntityFactory.CreateAsync(Constants.EntityDefinitions.MAsset) :
            existingAsset;

        // Set properties and relations
        if (!isAdaptation)
        {
            var culture = await _mClientFactory.Client.Cultures.GetDefaultCultureAsync();
            await SetAssetProperties(assetEntity, existingAsset, culture);
            await SetAssetRelations(assetEntity, existingAsset);
            targetAssetId = await _mClientFactory.Client.Entities.SaveAsync(assetEntity);
        }
        // await _mClientFactory.Client.Assets.FinalLifeCycleManager.ApproveAsync(assetEntity.Id.Value);

        var jobId = await CreateFetchJobAsync(targetAssetId, videoBlobUrl);

        _logger.LogInformation($"Fetch job created ({jobId}) for {targetAssetId} asset.");
    }

    private async Task<long> CreateFetchJobAsync(long assetId, string url)
    {
        var webFetchJob = new WebFetchJobRequest("Fetches rendered video from the storage.", assetId)
        {
            Urls = new List<Uri> { new Uri(url) }
        };

        return await _mClientFactory.Client.Jobs.CreateFetchJobAsync(webFetchJob);
    }