[OLD] [Mostly addressed] Documentation Refactor Feedback

[p-a-a-a-trick]

update 20230914

This has been addressed with #6911 - please create a new discussion for documentation feedback. Thanks!

Feedback Distillation

I spoke with a handful of users and obtained some feedback on the Ibis documentation.

User experience with Ibis ranged from new to contributing. Here is a summary of what was discussed:

• Contribution is challenging
• Messaging is more limited than it could be
• Accessibility to the package is limited to experienced python developers or developers already familiar with Ibis
• Documentation is difficult to navigate
• Lack of examples beyond the tutorial
• Lack of public roadmap

Proposal(s)

That said, here are some suggestions that came up that might help with these issues:

• Main proposal: refactor the docs to conform to the Diátaxis framework
• Corollary proposal: Ensure the docs can be used, in some way, as a language reference manual
• Side proposal (this idea was brought up in discussion and I thought it was a good one): Use a web-hosted, pre-built playground (e.g. Binder) to host examples or tutorials so users can have a boilerplate environment that they can use to toy around with Ibis without dealing with setup on their machine

To elaborate on the feedback above, see below. These are more notes from what I've gathered (mixed in with some of my own thoughts), so feedback on the feedback or pleas for clarification are welcome.

Contribution is challenging

First, there is a short and somewhat useful guide to setting up a dev environment and making contributions, but there isn't really any guidance on the package structure. If someone wants to implement a new backend or extend and existing backend, how do they do that? What should they avoid doing? What's been done before and then fixed to make better? All of this information can be found in the code, but is that really the best way to handle this?

Second, a common opinion was that a goal should be to make contributions as easy as possible. This way users that want to extend a backend or implement a simple fix can jump right in and do it. As it stands, a user needs to find the context necessary to not write clunky, bad, or pointless code. It's intimidating and requires a lot of spike work to read through the code base, so users are less inclined to make simple contributions.

Under the Diátaxis framework, a combination of the four sections could be used as a way to ease users into making contributions. Tutorials can be used to get users familiar with the basics, How To guides can be used to demonstrate common usage of various Ibis functions, Explanations can be used to discuss how the package is structured, and References can be used as a language reference manual.

Messaging is more limited than it could be

There's a problem trying to figure out Ibis's purpose. Why would someone want to use Ibis?

Upon landing at the ibis-project home page, users are greeted with "Expressive analytics in Python at any scale". This doesn't quite capture many Ibis use cases and really only captures the "ibis on an elephant" metaphor if users already understand what the package does and what the metaphor means. Ibis can also be used to minimize code change when there's a backend migration and it can be used to prototype code on in-memory data sources before deploying that code to production against a more formal backend, but that's not clear from the given tagline.

Past that, the home page launches right into installation and basic usage. A package introduction does exists, but it has some problems. First, this location is okay, but users are required to spend time looking for the package introduction when it should be (and can be) immediately available. Second, it's more of a "first tutorial introduction" than it is a package overview, and users with easily demonstrable use cases that aren't solely focused on "make the backend do more work" or "analytics" or "SQL in python" won't really grasp what Ibis can do. Why not just use a simple connector and fstrings?

There's also no real explanation for why someone would want to use Ibis, as if the project lacks purpose. There's some discussion about Ibis vs. SQL in the Introduction to Ibis page, and it actually explains one of Ibis's big draws (engine-agnostic queries!† / †terms/conditions apply), but this page isn't located somewhere a new (and arguably impatient) user would be able to find it. This page also doesn't really outline other strong abstract use-cases (migration, prototyping, [ironically] backend heavy-lifting)

The pandas docs have a dedicated section to the package overview under the Tutorials section. Personally I think information should be more front-and-center (e.g. linked or referenced on the home page), but it's really easy to find if you're a new user ("Getting Started" / "New to pandas?" -> left menu -> Package overview; good enough). A user should be able to find this information with very little effort on their part.

Right now, on the Ibis docs, users needs to be curious enough to actually click "Tutorial". This is okay with respect to availability, but that could be better--my opinion is this information should be obviously accessible from the home page. New users that don't know what Ibis is should be funneled directly to it.

A solution similar to how pandas handles it can be taken under the Diátaxis framework--funnel new users to the tutorials (and/or have a direct link to or front-page summary of what Ibis is).

Accessibility to the package is limited to experienced python developers or developers already familiar with Ibis

As I mentioned above, the consensus when gathering feedback was that the tagline and introduction needs a bit of work. It requires a bit too much thought to capture people that aren't necessarily interested in analytics. They might be at some point, but if their purpose isn't "analytics" and they don't know what Ibis is and they are short on time, then they're more likely to dismiss Ibis as a solution, and that can all be mitigated by simply providing a different (more holistic) tagline and a package overview containing more use cases that's immediately available.

Documentation is difficult to navigate

On that note, it's difficult to figure out where you should go when referencing docs and if they'll be useful to you given your experience level with Ibis. Examples, syntax, and common uses are bunched up inside of the tutorials (or drowned in the Backend Reference section), so finding simple solutions to simple problems can become a slog (and if such solutions exist, then they are easily missed).

The backend documentation is a bit overwhelming--it's more of a reference manual with sparse and inconsistent internal links and poor descriptions. It's a lot easier to figure out how something works or what something is if we can click on it and see its reference. Some parts of the docs have internal links, but more could be added. For example, if I was looking at the sql method under the Postgres backend and wanted to know what a TableExpression was, I would need to find where TableExpression lives because the description for the returned object is "Table expression". This would be fine if I could click ir.TableExpr and read the TableExpr reference/description, but right now it's circular and taunts me.

There isn't really a table of contents. Everything is based on the main side bar (left), sub-menus, and another side bar (right) to navigate sub-sections. There are a lot of things to click and it just feels clunky.

The documentation is littered with objects that aren't implemented or not typically appropriate for common use. If we ignore the format of the backend docs, this is fine as a reference manual so long as there are clear and consistent labels for what's implemented, or notifications for "use this if you want to do x", but users looking to solve simple problems will be overwhelmed.

Users shouldn't have to use a combination of the code base and a reference manual to figure out simple implementation. There should be self-contained guidance on how to do these things.

The Diátaxis proposal would funnel new users to a "Start Here", which would presumably be a "do once and never again", it would contain explanations and how to guides for new+ users, and it would contain a reference section for users looking to learn a bit more about what they're using as they use/improve it.

Lack of examples beyond the tutorial

As mentioned above, the tutorial should be a "do once and never again; here are other resources". It shouldn't be a go-to for examples or syntax reference.

Examples should be self-contained (or as self-contained as possible) and should be easily accessible. Users shouldn't have to scroll through a jupyter notebook to see an example on how to groupby an aggregate, and they certainly shouldn't have to go through trial+error on a common use case until they get it right.

The Diátaxis framework was designed to solve this problem--users looking for quick, common uses can go to the How To guides, and if they want to know more they can go to the Explanation or reference guides.

Lack of public roadmap

It would probably help to be a bit more clear about what's available now (via pip install or conda install or whatever install) and what's in the works for the future. Issues are given release milestones, but it might be more beneficial to the project if there was a consolidated list of planned features/bugfixes (and recent features/bugfixes) to help users figure out if Ibis is (or will be) good for their project(s).

End

Thanks, and let me know if you have any questions/comments/feedback/corollaries/ideas.

[jcmkk3]

Use a web-hosted, pre-built playground (e.g. Binder) to host examples or tutorials so users can have a boilerplate environment that they can use to toy around with Ibis without dealing with setup on their machine

This does seem like a direction that many libraries are pursuing recently. I think that JupyterLite is very promising as an alternative to binder.

One feature that I've always wished for in Ibis is a group of ibis.read_* functions that would default to sqlite or something. That would make it really easy to get up and running for a quick ad-hoc analysis of local data, as well as make docs/tutorials even more simple.

[gforsyth]

Hey @jcmkk3 -- could you elaborate a bit on the read_* functions? Would you want an ibis.connection object returned? A specific table from the DB? We can definitely add some convenience methods.

[jcmkk3]

Yes. Basically like if I were working locally in pandas or dplyr. I could use pokemon = ibis.read_csv('pokemon.csv') and it would return a connection.table object to a sqlite table. I'm not sure if it would be better to register it as a virtual table or load it into a temp sqlite database. There could be many ways to expand on it like having a plural version of the command(s) read a list of files and return a connection object instead. That could diverge too far from the purpose of ibis, however.

[p-a-a-a-trick]

@jcmkk3 the pandas backend is somewhat useful for that, but yeah there's no great guide on it (nor are there convenience wrappers for it):

import ibis
import pandas as pd

ibis.options.interactive = True

conn = ibis.pandas.connect({'foo': pd.read_csv('foo.csv')})

foo_table = conn.table('foo')

From there you can basically mock your csvs (or excel worksheets) as tables and then perform normal operations on them. In the event you want to move your code to prod/a different backend, just change the connector (and then edit anything that isn't supported in the new backend).

edit: my point for saying this is this information isn't immediately available and is a powerful use case for Ibis. Something like this doesn't really have a place in the docs as they are and, under the Diátaxis framework, would wind up in a how to guide or something.

[cpcloud]

While unrelated to documentation per se, we now have ibis.read_* for a few values of * (CSV, Parquet, JSON, Delta Lake).

The backend used to provide this functionality is DuckDB by default. However, the default backend can be configured by running:

ibis.set_backend("polars")  # set to the polars backend

con = ibis.connect("snowflake://...")  # set to a snowflake backend instance
ibis.set_backend(con)

[thegreymatter]

May I say the refreshed docs site is amazing? Big kudos.

[cpcloud]

@p-a-a-a-trick Thoughts on closing this or updating the description?

[NickCrews]

I think most things are addressed, but a roadmap would be nice. A pinned github issue/discussion, with a link from the concepts tab of the docs?

[p-a-a-a-trick]

Yep, docs are always a WIP, but agreed—most of these things have been addressed.

+1 on roadmap and let's close this bad boy

[cpcloud]

Thanks!