Applying tdd on select-only exercise

[blackk-foxx]

An alternative to #218, which uses Python to evaluate the test results.

Advantages:

• Simple, flexible, and easy to maintain
• Easy to report actual/expected diffs

[blackk-foxx]

The diff output is certainly not user-friendly. Instead of diff, we could run a custom Python script here that would generate a friendlier diff.

[blackk-foxx]

Either way, note that this would work with the existing test runner.

[IsaacG]

I'm not a fan of the multi-file approach.

If we introduce UPDATE, even if we're initially very light on details, we can use the existing framework which works quite well.

[blackk-foxx]

The latest version has the user enter all SELECT statements in a single file and evaluates the results in a single output file. Running sqlite3 < intro-select_test.sql generates a results.json with the content shown below. The message and output values are currently in JSON format, but we can easily convert them to a more user-friendly textual representation like the .mode columns format generated by sqlite.

{
  "version": 3,
  "status": "fail",
  "tests": [
    {
      "description": "ALL records => SELECT * FROM weather_readings",
      "status": "pass"
    },
    {
      "description": "Just location and temperature columns => SELECT location, temperature FROM weather_readings",
      "status": "pass"
    },
    {
      "description": "Without \"FROM\" => SELECT 'Hello, world.'",
      "status": "fail",
      "message": "Expected: [{\"'Hello, world.'\": \"Hello, world.\"}]",
      "output": "Actual: [{\"say_hi\": \"Hello, world.\"}]"
    },
    {
      "description": "All records from Seatle location => SELECT * FROM weather_readings WHERE location = 'Seattle'",
      "status": "pass"
    },
    {
      "description": "All records where humidity in range => SELECT * FROM weather_readings WHERE humidity BETWEEN 60 AND 70",
      "status": "pass"
    },
    {
      "description": "Just location column => SELECT location FROM weather_readings",
      "status": "pass"
    },
    {
      "description": "Only unique locations => SELECT DISTINCT location FROM weather_readings",
      "status": "pass"
    }
  ]
}

[blackk-foxx]

For some reason, this comes out as 57.79999999999 instead of the original 57.8 in the input data (see data.csv). Any ideas from the SQLite gurus on how to make it come out as 57.8?

[IsaacG]

Floating point numbers are hard for computers.

SQLite promises to preserve the 15 most significant digits of a floating point value.
ref

Numbers like 57.8 (or 0.8) cannot actually be handled very well by computers using the common representations. The original only looks like 57.8 when input or rounded. Float comparisons generally go better when you compare that the two values are close enough.

[IsaacG]

You can bypass this problem by picking nicer values, eg fractions which can be represented as a / (2^b).

sqlite> .mode json
sqlite> SELECT 57.8;
[{"57.8":57.79999999999999716}]
sqlite> SELECT 57.5;
[{"57.5":57.5}]

[IsaacG]

Adding a Python dependency to the test runner makes the image much, much larger. Without Python, apk stats says it's 12MB installed. With python3 that number goes up to 55MB.

I believe much of what you're doing in Python can be done pretty easily using jq. You could also use, say, a bash script plus jq to simplify the tests, eg extract the n'th item from the output JSON and the expected JSON for comparison. --slurp can be helpful here. Possibly also --slurpfile.

Using the test_data.json above and a .mode json style JSON out file,

# out
[{"date":"2025-10-22","location":"Portland","temperature":53.1,"humidity":72},{"date":"2025-10-22","location":"Seattle","temperature":56.2,"humidity":66},{"date":"2025-10-22","location":"Boise","temperature":60.4,"humidity":55},{"date":"2025-10-23","location":"Portland","temperature":54.6,"humidity":70},{"date":"2025-10-23","location":"Seattle","temperature":57.79999999999999,"humidity":68},{"date":"2025-10-23","location":"Boise","temperature":62.0,"humidity":58}]
[{"location":"Portland","temperature":53.1},{"location":"Seattle","temperature":56.2},{"location":"Boise","temperature":60.4},{"location":"Portland","temperature":54.6},{"location":"Seattle","temperature":57.79999999999999},{"location":"Boise","temperature":62.0}]
[{"'Hello world.'":"Hello, world."}]
[{"date":"2025-10-22","location":"Seattle","temperature":56.2,"humidity":66},{"date":"2025-10-23","location":"Seattle","temperature":57.79999999999999,"humidity":68}]
[{"date":"2025-10-22","location":"Seattle","temperature":56.2,"humidity":66},{"date":"2025-10-23","location":"Portland","temperature":54.6,"humidity":70},{"date":"2025-10-23","location":"Seattle","temperature":57.79999999999999,"humidity":68}]
[{"location":"Portland"},{"location":"Seattle"},{"location":"Boise"},{"location":"Portland"},{"location":"Seattle"},{"location":"Boise"}]
[{"location":"Portland"},{"location":"Seattle"},{"location":"Boise"}]

# test.jq
def range(upto):
  def _range:
    if . < upto then ., ((.+1)|_range) else empty end;
  0 | _range;

[
  range($want[0]|length) as $number
  | $want[0] | .[$number | tonumber].expected as $expected
  | $want[0] | .[$number | tonumber].description as $description
  | $out|.[$number | tonumber] as $got
  | if $got != $expected then {$description, $expected, $got, "status": "fail"} else {$description, "status": "pass"} end
]

» jq -c --slurpfile out out --slurpfile want test_data.json -n -f test.jq 2>&1
[
  {"description": "ALL records => SELECT * FROM weather_readings", "status": "pass"},
  {"description": "Just location and temperature columns => SELECT location, temperature FROM weather_readings", "status": "pass"},
  {
    "description": "Without \"FROM\" => SELECT 'Hello, world.'",
    "expected": [{"'Hello, world.'": "Hello, world."}],
    "got": [{"'Hello world.'": "Hello, world."}],
    "status": "fail"
  },
  {"description": "All records from Seatle location => SELECT * FROM weather_readings WHERE location = 'Seattle'", "status": "pass"},
  {"description": "All records where humidity in range => SELECT * FROM weather_readings WHERE humidity BETWEEN 60 AND 70", "status": "pass"},
  {"description": "Just location column => SELECT location FROM weather_readings", "status": "pass"},
  {"description": "Only unique locations => SELECT DISTINCT location FROM weather_readings", "status": "pass"}
]

My main worry here is with lining up the various outputs vs tests in a list. It's not obvious that the SELECT statements must be in a specific order. If a test fails, mapping the test to the SELECT is tricky.

I think this might be viable, but we'd also need some additional guardrails.

• A check added to validate the number of entries in the two files match. If the solution has more items than expected, do not run. If the solution has n items which is fewer than the tests, either do not run or run the first n tests.
• A test description must be written with the student as the primary audience (in contrast to the descriptions found in many of the problem specs). The test descriptions should also show up as a comment in the stub file. I'm tempted to suggest these should be numbers. That way it's simple to look at the test output and match it to a part of the solution.

-- Do not remove the comments! The comments line up with the test outputs.
-- Write your solution to each task beneath the comment.

-- 1: Select all records.

-- 2: Select the location and temperature.

-- 3: Say Hello!

...

[blackk-foxx]

@IsaacG, thanks for the jq prototype. I think it looks promising! I will use your code as a starting point, and work on adding the guardrails you mentioned.

The test descriptions should also show up as a comment in the stub file. I'm tempted to suggest these should be numbers. That way it's simple to look at the test output and match it to a part of the solution.

I agree, and I think the numbers should correspond to individual tasks within the exercise, like the tasks in exercises such as the Role Playing Game in the Elm track (among many others).