New Python package

[masaccio]

compact-json is falling further and further behind the latest version of this project and also suffers from performance problems given it's pure python (see my issue #35).

Given that Python.NET (pythonnet) is well advanced, I think the right solution is to offer a Python package that wraps the .NET classes and exposes a pythonic interface.

I've proven this out with a prototype and reasonably straightforwardly you can achieve an interface like this:

from fractured_json import CommentPolicy, EolStyle, Formatter, FracturedJsonOptions

with open("test.json", "r") as f:
    input_json = f.read()

options = FracturedJsonOptions(
    max_total_line_length=120,
    indent_spaces=2,
    max_compact_array_complexity=2,
    comment_policy=CommentPolicy.PRESERVE,
    json_eol_style=EolStyle.CRLF,
)
formatter = Formatter(options=options)

output_json = formatter.reformat(input_json)
print(output_json)

The code behind this dynamically looks up enum values and the public interface of FracturedJsonOptions so any new options will automatically be picked up. In my original port, I changed max_compact_array_complexity to max_compact_list_complexity but that's arguably too far and the new implementation will completely break the API for the original anyway.

In principle I could just create a PyPI package called fractured_json and the appropriate build infrastructure in GitHub to create updates whenever a new release happens here. More logically though, the builds here should create a Python package and publish it to PyPI.

The runtime requirements for .NET in Python are not super simple so I can see that python purists might stumble over things like environment variables and keeping the .NET runtime up-to-date so there may be support involved in this.

So this is an open question to how @j-brooke you'd like to handle this. I am perfectly OK with handling the Python packaging and that's likely easiest for you, especially if you'd rather not learn yet another environment. I don't have access to any Windows hosts other than the GitHub runners so I might have to lean on you occasionally if that goes wrong.

[masaccio]

init.py

Adding the implementation here. A good bit of this is vibe-coded so I'd very much want to clean this up and also build test infrastructure to be confident that it handles all the options correctly. So far I've done nothing more than run code like the example above.

[j-brooke]

I like this idea a lot. They Python ecosystem is hot these days, and I'd love for FJ to be available there. But I'm a noob there, so I'm glad you know your way around, @masaccio. I'd absolutely love it if you would maintain the Python side.

I'm currently getting ready to release version 5. All the code and tests are done and checked into the main branches for .NET and TypeScript. The change logs are updated. But I still need to update a bunch of documentation. So version 5 would be a good place to start, I expect.

A few things we probably should consider:

1. Features. I believe compact-json has a couple features that FJ doesn't. We should probably look at those and think about which should be brought into FJ (and how soon), and which should be left out. (I believe the "compact multiline array" concept was expanded to objects, for instance.)

2. Command line usage. I believe that compact-json's CLI implementation sees a good amount of use. The .NET CLI app, as far as I know, hasn't. It's available only as C# source code right now, although I guess there's a way to distribute it as a global tool. We should probably think about which to support, and if both, think about reconciling their behaviors. (I've made changes pending for version 5, allowing the use of a config file for the .NET CLI app.)

3. Wide character support. A version or two I removed baked-in wide character support for the library versions. The caller is able to provide their own string length function, using whatever wide-character library they prefer. (The VSCode extension, .NET CLI app, and web formatter still have fixed dependencies.) Is this going to cause complications for the Python wrapper approach?

Let me know what you think, @masaccio and anyone else who wants to pitch ideas.

[masaccio]

The python interface for .NET seems to be really very good so I expect that callbacks will work for wide character support. I have that in compact-json today.

For version 5, are any API changes already in as I can test against your dev branch. What I was thinking for the python package was to version it and build it using triggers off your releases so as long as tests keep passing it will always be-up-to date. I can easily convert enums to command line options and the only manual thing I’ve spotted is free-text descriptions of options.

I will take a look at the config file too for the CLI as not everyone wants a big long list of verbose command line options.

Let me build something and see what gotchas I find. Testing might be the significant work unless I come up with a way to with reflection to run the .NET tests on the python library. I already use the reflection APIs and they too seem remarkably complete.

[j-brooke]

The CHANGELOG in the main branch shows the changes coming for version 5. In brief: alignment for expanded object properties, 2 new options, a few removed, and a few defaults changed.

[masaccio]

The python wrapper will live here: https://github.com/masaccio/fractured-json-python

The CLI seems to work but there's a lot of testing and automation to do get before I publish on PyPI.

[j-brooke]

Excellent. I updated the FJ wiki page for Python to link to both versions. Please let me know if you have edits. I'll update links to point to that instead of compact-json so people can get the whole picture. That includes the main FJ wiki page and the web formatter's header. There may be other places that need updating too.

[masaccio]

For anyone tracking, the package has now been published on PyPI (https://pypi.org/project/fractured-json/). I consider the options management feature complete though the Enums could do with some refactoring to be more Pythonic.

Serialisation is not currently supported, but that's pretty much all that is missing.

Project-wise it needs work for coverage, especially around exceptions, and I've not done any GitHub workflow hooks to have it build off changes to the upstream FracturedJson.

I've also marked compact-json as archived so it should be clear it gets no new development.

[j-brooke]

Excellent. I'll update the docs over here.

By the way, the documentation and repository links on the PyPl page give me 404's. I gather that https://github.com/masaccio/fractured-json-python is the permanent repo URL.

[masaccio]

Thanks for spotting that. "fractured-json-python" is indeed the correct repo name. I wanted to distinguish it clearly from yours whilst keeping the Python package name sensible.