Lua error with several fonts and complex script

[jbezos]

Consider the following MWE:

\documentclass{article} 

\makeatletter
\directlua{
function test (head)
  texio.write_nl('==== ')
  texio.write_nl('')
  for item in node.traverse(head) do
    if item.id == node.id'glyph' then
      texio.write(string.format(' [\@percentchar X]', item.char))
      texio.write('<' .. item.font .. '>')
    else
      texio.write(' [' .. item.id .. ':' .. (item.subtype or '?') .. ']')
    end
  end
  return head
end
luatexbase.add_to_callback("pre_linebreak_filter", test, "testv", 0)
}
\makeatother

\usepackage{fontspec}

\setmainfont{FreeSerif}
\newfontfamily\hindifont{FreeSans}[Renderer=Harfbuzz,Script=Devanagari]

% \usepackage{unicode-math}

\begin{document}

\(A\) {\hindifont आव्यूह} ABC {\hindifont हैं}?

\end{document}

The PDF is generated correctly, and the following is shown in the log:

 [9:?] [0:3] [11:0] [41]<40> [11:1] [12:13] [906]<50> [121417]<50> [92F]<50> [94
2]<50> [939]<50> [12:13] [41]<27> [42]<27> [43]<27> [12:13] [939]<50> [12145A]<5
0> [3F]<27> [14:2] [12:15]

It’s a representation of the node list (in the form [node type:sybtype] or [char]<font>) . [121417]<50> comes from [935]<50> [94D]<50>, which are ’ligated’.

Now, uncomment \usepackage{unicode-math}. The following error is raised:

! error:  (linebreak): invalid list tail, probably missing glue
!  ==> Fatal error occurred, no output PDF file produced!

And the node list becomes:

==== 
 [9:?] [0:3] [11:0] [1D434]<41> [11:1] [12:13] [906]<53> [121417]<53> [92F]<53> 
[942]<53> [939]<53> [12:13] [41]<27> [42]<27> [43]<27> [12:13] [939]<53> [12145A
]<53> [3F]<27> [14:2] [12:15] [94D]<53>

It’s almost the same, except for the math A ([41]<40> becomes [1D434]<41>), which is fine, BUT, after the font is processed, [94D]<53> pops up as the last node, after the glue ending the paragraph.

I’m not sure why, but apparently the last node in a different font (with ‘?’) is triggering the error.

See https://tex.stackexchange.com/questions/741095/i-am-getting-the-error-message-error-linebreak-invalid-list-tail-probabl.

[ponte-vecchio]

Interestingly, with the following (even more minimal) example, there is no issue:

\documentclass{article} 

\makeatletter
\directlua{
function test (head)
  texio.write_nl('==== ')
  texio.write_nl('')
  for item in node.traverse(head) do
    if item.id == node.id'glyph' then
      texio.write(string.format(' [\@percentchar X]', item.char))
      texio.write('<' .. item.font .. '>')
    else
      texio.write(' [' .. item.id .. ':' .. (item.subtype or '?') .. ']')
    end
  end
  return head
end
luatexbase.add_to_callback("pre_linebreak_filter", test, "testv", 0)
}
\makeatother

\usepackage{fontspec}

\setmainfont{FreeSerif}
\newfontfamily\hindifont{FreeSans}[Renderer=Harfbuzz,Script=Devanagari]
\usepackage{unicode-math}

\begin{document}

- \(A\) {\hindifont आव्यूह} ABC {\hindifont हैं}?
+ $A$ {\hindifont हैं}?

\end{document}

which produces the following nodelist:

====
 [9:?] [0:3] [11:0] [1D434]<41> [11:1] [12:13] [939]<53> [12145A]<53> [3F]<27> [
14:2] [12:15]

I suspect the use of different fonts between हैं and ? isn't the sole explanatory factor here.