Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cloudflare Data Connector - Chunking of file causing an exception #9462

Closed
katerobson08 opened this issue Nov 23, 2023 · 2 comments
Closed
Assignees
Labels
Connector Connector specialty review needed

Comments

@katerobson08
Copy link

Describe the bug
**One of our customers has an issue with the Cloudflare function app falling over on files larger than 2MB. The customer file was 4.2MB.

The function app raises the following exception:**

Exception: HttpResponseError: Download stream interrupted.
Stack: File ""/azure-functions-host/workers/python/3.8/LINUX/X64/azure_functions_worker/dispatcher.py"", line 402, in _handle__invocation_request
call_result = await self._run_async_func(
File ""/azure-functions-host/workers/python/3.8/LINUX/X64/azure_functions_worker/dispatcher.py"", line 654, in _run_async_func
return await ExtensionManager.get_async_invocation_wrapper(
File ""/azure-functions-host/workers/python/3.8/LINUX/X64/azure_functions_worker/extension.py"", line 147, in get_async_invocation_wrapper
result = await function(**args)
File ""/home/site/wwwroot/AzureFunctionCloudflare/main.py"", line 69, in main
await asyncio.gather(*cors)
File ""/home/site/wwwroot/AzureFunctionCloudflare/main.py"", line 148, in process_blob
async for chunk in blob_cor.chunks():
File ""/home/site/wwwroot/.python_packages/lib/site-packages/azure/storage/blob/aio/_download_async.py"", line 151, in anext
self._current_content = await self._iter_downloader.yield_chunk(chunk)
File ""/home/site/wwwroot/.python_packages/lib/site-packages/azure/storage/blob/aio/_download_async.py"", line 62, in yield_chunk
return await self._download_chunk(chunk_start, chunk_end - 1)
File ""/home/site/wwwroot/.python_packages/lib/site-packages/azure/storage/blob/aio/_download_async.py"", line 105, in _download_chunk
chunk_data = await process_content(response, offset[0], offset[1], self.encryption_options)
File ""/home/site/wwwroot/.python_packages/lib/site-packages/azure/storage/blob/aio/_download_async.py"", line 27, in process_content

I got around this by changing the MAX_CHUNK_SIZE_MB parameter to 5MB (to be bigger than the file so that it didn't try to chunk it)

To Reproduce
Steps to reproduce the behavior:

  1. Create a file larger than 2MB in storage account
  2. Run Cloudflare function app
  3. Fails with exception produced: Exception: HttpResponseError: Download stream interrupted.

Expected behavior
I would have expected the file to have been proccesed and then deleted.

Screenshots
N/A

Desktop (please complete the following information):

  • OS:
  • Browser:
  • Version:

Smartphone (please complete the following information):

  • Device: [e.g. iPhone6]
  • OS: [e.g. iOS8.1]
  • Browser [e.g. stock browser, safari]
  • Version [e.g. 22]

Additional context

Copy link
Contributor

Thank you for submitting an Issue to the Azure Sentinel GitHub repo! You should expect an initial response to your Issue from the team within 5 business days. Note that this response may be delayed during holiday periods. For urgent, production-affecting issues please raise a support ticket via the Azure Portal.

@v-sudkharat v-sudkharat added the Connector Connector specialty review needed label Nov 24, 2023
@v-sudkharat
Copy link
Contributor

v-sudkharat commented Nov 24, 2023

Hi @katerobson08, Hope you are doing well. This solution is supported with Cloudflare, we kindly request you to raise an issue ticket with Cloudflare's support team.
Sharing the Cloudflare mail ID and link -
Mail Id - "support@cloudflare.com"
Link - "https://support.cloudflare.com/"
So, closing this issue. If you still need support for this issue, feel free to re-open it any time. Thank you for your co-operation.
Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Connector Connector specialty review needed
Projects
None yet
Development

No branches or pull requests

3 participants