Input format should accept unflattened as well as flattened data, but also a list of IDs for basic counts (ids encode a timestamp so we can give back counts over time for a list of IDs only)
Ideally we should also support compressed files, and a directory of files. Eg: your dataset can be is 1 big xz compressed file with 1 tweet per line, or a directory of gz files with 1 file per hour or per day, etc.
It would also be nice to support v1.1 format tweets but this would be a lower priority than v2 formats.
Input format should accept unflattened as well as flattened data, but also a list of IDs for basic counts (ids encode a timestamp so we can give back counts over time for a list of IDs only)
Ideally we should also support compressed files, and a directory of files. Eg: your dataset can be is 1 big xz compressed file with 1 tweet per line, or a directory of gz files with 1 file per hour or per day, etc.
It would also be nice to support v1.1 format tweets but this would be a lower priority than v2 formats.