Skip to content

fix: stream files too large to read unbuffered#107

Open
vibhor-aggr wants to merge 1 commit into
koajs:masterfrom
vibhor-aggr:fix/skip-large-unbuffered-read
Open

fix: stream files too large to read unbuffered#107
vibhor-aggr wants to merge 1 commit into
koajs:masterfrom
vibhor-aggr:fix/skip-large-unbuffered-read

Conversation

@vibhor-aggr

@vibhor-aggr vibhor-aggr commented Jun 11, 2026

Copy link
Copy Markdown

Closes #103.

When fs.readFileSync() reports ERR_FS_FILE_TOO_LARGE, unbuffered mode should still be able to serve the file through the existing stream path instead of crashing while trying to compute the eager MD5.

This preserves the current eager hash behavior for ordinary files, rethrows other read failures, and still rethrows the too-large error when options.buffer is enabled because that mode requires the file in memory.

Verification:

  • ./node_modules/.bin/mocha --grep 'unbuffered read is too large'
  • npm test
  • git diff --check

Summary by Sourcery

Handle large files in unbuffered mode without crashing by falling back to streaming when eager reads exceed size limits.

Bug Fixes:

  • Avoid crashes when fs.readFileSync throws ERR_FS_FILE_TOO_LARGE in unbuffered, dynamic mode by skipping eager buffering and MD5 computation.

Tests:

  • Add a regression test to ensure dynamic unbuffered responses stream correctly when readFileSync reports files as too large.

@coderabbitai

coderabbitai Bot commented Jun 11, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@vibhor-aggr, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 1 minute and 30 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more credits in the billing tab to continue.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: bfbfdefe-8cb3-415c-9571-9ffdaaf8d840

📥 Commits

Reviewing files that changed from the base of the PR and between 182f260 and b1438ac.

📒 Files selected for processing (2)
  • index.js
  • test/index.js

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@sourcery-ai

sourcery-ai Bot commented Jun 11, 2026

Copy link
Copy Markdown

Reviewer's Guide

Handles ERR_FS_FILE_TOO_LARGE when loading static files so unbuffered, dynamic mode falls back to streaming instead of crashing, and adds a regression test for this behavior.

File-Level Changes

Change Details Files
Handle ERR_FS_FILE_TOO_LARGE when eagerly reading files so unbuffered dynamic mode can still stream large files while buffered mode continues to require fully reading the file.
  • Wrap fs.readFileSync in a try/catch when loading files to intercept read errors
  • Suppress ERR_FS_FILE_TOO_LARGE only when options.buffer is false, otherwise rethrow the error
  • Skip MD5 hash computation when the file buffer is unavailable, preserving existing behavior for successfully read files
index.js
Add regression test to simulate too-large unbuffered reads and assert that files are still streamed successfully in dynamic mode.
  • Introduce mz/fs and os dependencies in the test suite for mocking readFileSync and creating temporary directories
  • Add a test case that creates a temporary large file, monkey-patches mzfs.readFileSync to throw ERR_FS_FILE_TOO_LARGE for that file, and verifies a 200 response with the expected body
  • Ensure cleanup restores the original readFileSync and removes temporary files and directories
test/index.js

Assessment against linked issues

Issue Objective Addressed Explanation
#103 Prevent the application from crashing with RangeError[ERR_FS_FILE_TOO_LARGE] when serving files larger than 2GB in unbuffered mode (buffer: false), and allow those files to be streamed instead.
#103 Preserve existing behavior for other read errors and when buffer: true (i.e., still throw on ERR_FS_FILE_TOO_LARGE when buffering is required).

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've reviewed your changes and they look great!


Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug] RangeError [ERR_FS_FILE_TOO_LARGE] when buffer set false.

1 participant