Arabic support for Unicode 17.0 would be appreciated

[LornaSIL]

I don't know if Harfbuzz needs updating, or your support for Unicode 17.0 needs updating, but I'm not getting the proper rendering for an Arabic script character that was encoded in Unicode 17.0

This is the text: ࢏ ࢏࢏َ࢏

It should look like this (LibreOffice and Firefox):

In ptxPrint 3.0.38 (and 3.0.9), the characters don't shape at all:

Font Scheherazade New v4.500: https://software.sil.org/scheherazade/download/
It would be great to get this working.

[LornaSIL]

@devosb has told me "I think PTXprint is using TeX Live 2023 (on windows), so whatever version of HarfBuzz is in that version of TeX Live would not have been updated for Unicode 17."

At the encouragement of @devosb I have updated my version of TeXLive from 2025 to 2026. The above now renders correctly in XeTeX. When I do xetex --version, it says Compiled with HarfBuzz version 12.3.2; using 12.3.2

I don't know how to update my version of xetex in ptxPrint, but I suspect that's what needs updating before it will render the Unicode 17.0 characters. I hope you can do that.

[mhosken]

This involves an update to libharfbuzz and / or xetex.exe. A workaround is to get an updated harfbuzz.dll and put it somewhere in your path where xetex can find it and xetex should run just fine with the new Unicode properties

[davidg-sil]

I can confirm that (at least a year or 2 ago) TeX reporting those 2 numbers separately was not just cosmetic; TeX seemed entirely happy to use a much more modern version of the harfbuzz library than it was compiled with.

[devosb]

@davidg-sil I am curious in what environment you saw two different versions (Compiled with and using) of HarfBuzz being reported. I have seen this as well on Ubuntu, using TeX Live (TL) as packaged by Debian/Ubuntu. But xetex from the TL installer (both on Ubuntu and Windows) seems to statically link HarfBuzz, so I would not have thought that a newer harfbuzz.dll would have been loaded.

Using ldd on MSYS2 shows that xetex.exe is a dynamic executable (as well as xetex.dll) but I don't see where HarfBuzz is linked in either TL 2023 (which is what PTXprint is currently using on Windows) nor on TL 2026 (which I have ready to commit).

In the past I have used (IIRC) dumpbin.exe /imports from Microsoft Visual Studio (which I don't have installed at the moment) to see what the shared libraries are.

As an aside, you can see that the ICU DLL is linked to from both xetex.exe and xetex.dll.

[davidg-sil]

It was on linux; I was trying different versions to find out why we were getting different output depending whose machine it was on.... I don't think I've compiled anything for an MSoft operating system since windows3.1.

[devosb]

I tried putting all the files that match libharfbuzz*.dll from the 14.4.0 HarfBuzz release into the same folder as xetex.exe, but they do not seem to be used (xetex -version stays the same).

So I can push the updated TL Windows files, I was hoping to see that the updated TL fixes the issue before pushing, but I think the update should work in general. I did not want to push right before a release, so that there would be adequate time to test.

I don't know why @LornaSIL is getting correct shaping with TL 2026, but the stripped down copy of TL 2026 that I am testing PTXprint with does not shape correctly (for the example in this issue, other Arabic text seems to shape correctly). I have set the font (Scheherazade New v4.400 and sometimes a local build of the font), the text direction (RTL), and the script (Arabic).

[davidg-sil]

@devosb, a quick duck-duck-go says there's some step needed to register .DLL files before they can be used. Did you do that step?

Also, with a few more searches, I see that windows has mechanisms that can pick which .DLL gets used for a binary (e.g. local/system dir/maybe more specific, even?) or even bind the DLL to the binary, I guess making it pseudo-static linked (why not create it that way if that's what you want??).
(I searched for "windows register .DLL", "windows bind .DLL to binary " and "windows restrict binary to specific .DLL")

[LornaSIL]

This involves an update to libharfbuzz and / or xetex.exe. A workaround is to get an updated harfbuzz.dll and put it somewhere in your path where xetex can find it and xetex should run just fine with the new Unicode properties

Neither one of these worked. I put the 2026 TeXLive version of xetex.exe in place of the existing one in ptxprint and it wouldn't run.

I restored to the prior version of xetex.exe and ptxprint runs (but of course the character I want isn't shaping)

I found where ptxprint has libharfbuzz-0.dll and libharfbuzz-gobject-0.dll and replaced both of those and ptxprint just gave a bunch of errors and wouldn't open.

I replaced libharfbuzz-gobject-0.dll with the original one and still got errors and it wouldn't open.

I replaced libharfbuzz-0.dll with the original one and ptxprint runs (but of course the character I want isn't shaping)

This is too fragile. It needs a proper fix.

[devosb]

I will try registering the DLL (and adding it to the PATH) but will be pleasantly surprised if that works, since I have not needed such steps in the past 10 years of updating XeTeX for ptx2pdf.

I think most of the functionality of xetex is in xetex.dll, not xetex.exe. In one of the previous upgrades of TeX Live, xetex.exe did not change at all, but xetex.dll did.

The various harfbuzz DLLs that @LornaSIL mentions are not in the ptx2pdf repo, so I am not sure how I would even upgrade them, or exactly where they come from, even if that was a solution.

In TeX Live 2026 that I downloaded the version of HarfBuzz is 12.3.2, which is newer than the version when HarfBuzz was updated for Unicode 17 (which was 11.5.0). So while it would be nice to be able to update HarfBuzz independently of TeX Live, I don't see how that would resolve this particular issue.

[devosb]

I was able to get PTXprint to shape new characters from Unicode 17. The updated files from TL 2026 are good, the needed change was to disable adjustments to spacing between letters (Fonts+Scripts tab / Spacing Adjustments group / Between Letters option). This option defaults to on, with values of 2.0 for Min/Shrink and 0.5 for Max/Stretch. Even with this option on, other characters seem to shape (but not the characters from Unicode 17 that this issue is about).

You can see the difference in the generated .tex file from PTXprint in a diff from meld, the option with space adjustment between letters (default is enabled) is on the left, having the option disabled is on the right.

[devosb]

Does mkptxunicode need to be run to update ptx-unicode.tex? Maybe after updating ucdata_pickle.bz2, and that file will need updating itself once Unicode 18 comes out later this month.

[mhosken]

I assume this isn't a production driven issue, so we can wait?

[LornaSIL]

There are users wanting to use this character and hardly anything is supporting it yet. If you support it, they will likely use ptxprint since they can't use InDesign.

[devosb]

@mnjames Did you have to disable intercharacter spacing like I had to in order to get shaping with the new character?

I am curious to see how close my work on updating XeTeX is to Claude's.