"Grover::JavaScript::RangeError: Invalid array length" when generating very large PDFs

[jfeust]

We generate large reports with potentially thousands of images. Once the reports get to be a certain size we get a Grover::JavaScript::RangeError: Invalid array length error thrown. These reports are in the range of 2000+ pages with 4000+ images.

Heroku
Grover 1.2.3
Browserless.io for processing

[abrom]

Hmm interesting problem. Hard to say where that's actually springing from without more details. I've created a branch on the repo called debug-js-error. Can you switch to that and repeat the test/failure. It will include the JS stack output in the message.
eg

> Grover.new('asd').to_pdf
/path/to/grover/lib/grover/processor.rb:94:in `call_js_method': page[(convertAction + "123")] is not a function (Grover::JavaScript::TypeError) 

TypeError: page[(convertAction + "123")] is not a function
    at _processPage (/path/to/grover/lib/grover/js/processor.cjs:318:59)
	from /path/to/grover/lib/grover/processor.rb:20:in `convert'
	from /path/to/grover/lib/grover.rb:51:in `to_pdf'
	from (irb):1:in `<main>'
	from bin/console:8:in `<main>'

(in the above example I intentionally added some dodgy script to cause the error, but you can see the JS stack is now included including a line number 😄)

I'll need to have a bit of a think about how this might integrate into the workflow in a more permanent way 🤔

[abrom]

note, I have a suspicion that this is actually coming out of the puppeteer code as the longest array used in Grover (directly) is 5 elements long.. and there is no direct indexing of elements...

[abrom]

Unless it is an array out of bounds error being misrepresented as an array length issue when converting the PDF output into the Buffer to pass back to ruby land 🤔

Anyway.. send through the JS trace 😄

[jfeust]

@abrom

WARN: Grover::JavaScript::RangeError: Invalid array length
RangeError: Invalid array length
   at Buffer.toJSON (node:buffer:1154:15)
   at JSON.stringify (<anonymous>)
   at .../.rbenv/versions/3.1.6/lib/ruby/gems/3.1.0/bundler/gems/grover-df954f3f4755/lib/grover/js/processor.cjs:357:35

[abrom]

ok, looks like you've come up against a limitation of the Buffer library. Should be an easy fix though. Just write the contents of your HTML to a file and pass a reference to that in. It does mean enabling the allow_file_uri option, so you'll want to be extra extra extra careful about trusting the HTML you're trying to render.. see https://github.com/Studiosity/grover?tab=readme-ov-file#allow_file_uris It allows any script to include/read contents of the filesystem with the same privs as the browser.

# config/initializers/grover.rb
Grover.configure do |config|
  config.allow_file_uris = true
end

...

pdf = Tempfile.create do |file|
  file.write html
  file.flush
  Grover.new("file://#{file.path}").to_pdf
end

[abrom]

Oh wait, sorry.. I misread your message.. it's failing to write the result.. not to parse the input.

That is a trickier one to solve..

I see two options

1. You pass in a file path to write the output of the conversion to, then catch/ignore the error thrown

begin
  pdf = Grover.new(html, path: some_temp_path).to_pdf
rescue Grover::JavaScript::RangeError
  pdf = File.read some_temp_path
end

Although this will almost certainly want to include some more checks to make sure things have actually been saved properly. eg check if the pdf starts with the PDF file header %PDF-<version> (will likely be %PDF-1.4)

2. Modify Grover to pass through an option telling it not to write the conversion output via stdout (using JSON.stringify/Buffer.toJSON):

const dontReturnResult = options.dontReturnResult; delete options.dontReturnResult;
const result = await page[convertAction](options);
if (dontReturnResult) return
return Buffer.from(result);

Then you'd use it as:

Grover.new(html, path: some_temp_path).to_pdf # this will now just return `nil`
pdf = File.read some_temp_path # So we need to read it in ourselves