Switch to Python 3

[paride]

Python 3 is now 10 years old, certainly mature enough to be the first choice for new projects. In the case of Hack switching seems rather easy:

• call python3 instead of python in the scripts;
• use pip3 instead of pip;
• drop the from future ... imports in the python scripts.

This could really be sufficient.

[anthrotype]

yes!
fontmake and all its dependencies are py2.py3 compatible. If you see any differences in the fonts generated with py2 and py3 than that's a bug and I'll gladly fix it.

[chrissimpkins]

Are you testing against both Py2 and Py3 in fontTools and fontmake (and by extension all of your dependencies...) Cosimo?

[anthrotype]

of course we are

[chrissimpkins]

@anthrotype OK thanks. Just confirming. I am building releases with pip3 installed build tools / Py3 so I can confirm that we have no problems with the current build on Py3.

[chrissimpkins]

I guess the drawback here is that we are asking users who don't necessarily have Python3 installed to do so in order to build Hack which is an unnecessary dependency for this project because it builds without issues under Python2 and Py2 remains the default interpreter installed on those platforms that install Python by default (including most Linux distros and macOS). Is there a project-specific benefit to Py3 here or is this a suggestion to participate in the movement to push Py3 as the default interpreter everywhere. The root of this problem is that python / pip are not mapped to the python3 interpreter and pip3 out there in the wild, not that we don't support Py3.

[chrissimpkins]

And understand I am in complete agreement with your opinion in general. Simply playing devil's advocate.

[anthrotype]

I think you're mixing up font users and devolopers.
Normally, "users" aren't expected to build anything (including fonts) from source, they are intrerested in using the end product and they can get it as part of the OS or with some package manager.

"developers" on the other end can easily install python 3 as a build requirement.

Python 3 is just python; python 2 is legacy python. In a couple of years upstream will drop support and it will no longer receive bugfixes.

All linux distros that I came across, ship some version of python3 alongside python2.
I'm not impressed by the fact that the generic python executable is not a symlink to python3 everywhere (yet). This is to avoid breaking so many scripts that are written for python2 but don't say it explicitly in their shebang.

It's increasingly becoming a burden for all of us that maintain python libraries to keep writing code that is compatible with both 2 and 3.

The Hack fonts project is not a "library"; it doesn't need to support any possible python runtime. You don't get a different output if you build the fonts with PyPy, do you?

[chrissimpkins]

"developers" on the other end can easily install python 3 as a build requirement.

Agree. But this isn't a pure Python source project, though it does use Python source as part of its build tooling. It is a typeface project built around UFO source files. There may be (I hope that there are...) typeface developers (broadly defined, let's use the name tinkerer since there is reasonable Python knowledge in much of the typeface development community given that many of the typeface tools are built with Python) out there who do not develop in Python, do not need Python3, don't know about the pip/pip3 issue yet may want to modify and build fonts from the source here, whether for upstream contributions or to maintain their own downstream source with any modifications to the design that they'd like to make using whatever editor they would like to use. We want to support this IMO. My goal has been to develop a scripted build strategy that is relatively transparent across all developers/users for as many types of build requirements as we can support so that no one is forced to build via a GUI font editor and can use any font editor that supports UFO source files. We don't have a good approach for Windows users at the moment. They have to install a number of significant dependencies to build the fonts. We can only do so much :) Our entire alternate glyph build strategy through the alt-hack repository is designed around empowering users to build their own modified fonts on the command line with the build tooling workflow that we've developed here.

I am not arguing against you about the benefit of the transition to Python 3 in general. It must happen out there and I would like to help support the transition here. I suppose I am arguing against a hard change to python3 use or a failed build if that is the suggestion. Unfortunately Py3 is just not a broad reality in the wild at this stage so this means that we introduce a new dependency to the project and potentially create obstacles to development of the typeface that are not necessary for the advancement of the typeface. When/if Cosimo deprecates and drops support for Py2 in the compiler and fontTools, we will promptly make an enforced move to Py3 for all :) If there is to be an impetus for this type of move across projects that do not need Py3 features, it will come from the libraries not the users of the libraries and I suspect that the libraries are waiting for the Python3 penetration to broaden before they are prepared to make such a move. It's a problem and it is not resolving quickly.

If it would work for both of you, I think that we can work around the issue by detecting python3 and pip3 in both the dependency install scripting and the execution of Python scripts during the build process, then default to Py3 when they are installed. How does that sound?

[anthrotype]

Sure, but then the burden will be on you to support building the Hack fonts with both py2 and py3. You'd have to test the output of both and make sure they produce identical output.

[chrissimpkins]

You just told me that you were testing against both ;)

[chrissimpkins]

And I am assuming that your versions follow SemVer. If not, we need to know this.

[anthrotype]

We are testing on both (and yes, we do follow semver). However coverage is not 100% everywhere, I'm afraid. And we do sometimes find small differences, mostly to do with the fact that in python3 unsorted collections like sets or dicts use a random hash seed that changes upon every run; if the code was originally written for python 2 (which is usually the case with old code) and it doesn't take that into account (e.g. when doing loops over unsorted collections, or when converting them to sequences like tuples or lists without a deterministic sorting), then you may get different (though functionally equivalent) output when running with py2 or py3.
I'm not aware of any such remaining differences myself, but we did come across a few of them.

[anthrotype]

This is mostly a concern for the libraries that we inherited, so to speak. All the new code that I merge is (should be) accompanied by its tests, and coverage cannot (ought not to) regress.

[chrissimpkins]

Why don't we set up a branch that tracks main and builds with Python3 only. We can help test that issue if that is helpful. It would be useful to know what we might expect to see as differences so that we know what to diff.

[anthrotype]

It would be useful to know what we might expect to see as differences

if I knew them, I would fix them :)
I'd watch for things that end up with a different order. This is mostly about reproducible builds rather than functionality, as sometimes the order doesn't matter, until it does.