|
myTinySQL v0.2.0-2-g5000098
A tiny SQL interpreter over fixed-width text files
|
This guide shows why myTinySQL works, not just that it does. Every check below is a command you can copy, followed by the result you should see. If you get the same result, you have checked that part yourself.
These are the same checks used while developing the project (on Ubuntu 26.04, with g++ 15.2), and every command and expected result in this file was run before it was written down.
One command runs checks 1 to 10 and 12, compares every result with the one this guide expects, and ends with a single verdict. First install the tools the checks use: a fresh Ubuntu has no curl, and without it the checks stop at once.
Then run all the checks:
You should see (output, don't type it) one line per check, then the verdict:
If something is wrong, the verdict is OOOPS - SOMETHING IS WRONG, the failed checks say why, and the script prints where its log is: everything each check printed, with the expected and actual results. If a tool is missing, it says which one and how to install it, before running anything.
make check is tests/check_all.sh; tests/check_all.sh --offline skips the two checks that need the internet (10 and 12).data/ is only read (check 7).The rest of this guide is the same checks one by one, with what each one shows and why it exists.
| # | Check | Command | Expected | What it shows |
|---|---|---|---|---|
| 1 | Clean build | make clean && make | no warnings | The compiler finds nothing suspicious |
| 2 | Automated tests | make test | 8 × PASS | The SQL gives exactly the expected output |
| 3 | Command line | echo "..." \| ./myTinySql ... | rows, errors, exit code | Scripts and error handling work |
| 4 | Browser server | curl ... | 200, 404, 405 | The server answers correctly |
| 5 | Idle connection | exec 3<>/dev/tcp/... | 200 OK after 6 s | Browsers that "preconnect" are served |
| 6 | Parallel writes | 20 × curl at once | 28 rows, no duplicate ID | Simultaneous requests can't corrupt a table |
| 7 | File integrity | awk on data/ | one length per file | Every row has the exact fixed width |
| 8 | Memory safety | sanitizer build | 0 problems | No memory errors or undefined behaviour |
| 9 | Documentation | make doxygen | no warnings | Every declaration is documented |
| 10 | Fresh install | git clone + make test | 8 × PASS | The project works from a clean copy |
| 11 | By hand, in a browser | checklist | as described | The web page works for a person |
| 12 | On GitHub | Actions tab | green check | The online documentation is up to date |
Each check has two kinds of grey boxes:
0 means the command must print 0.The build tools from INSTALLATION.md, plus the tools some checks use. A fresh Ubuntu has no curl, so install it even if you installed everything else. On Ubuntu:
| Tool | Used by | Note |
|---|---|---|
curl | checks 4, 5, 6, 12 | Sends requests to the server from the terminal; not installed on a fresh Ubuntu |
git | check 10 | Downloads a fresh copy of the project |
doxygen | check 9 | Only for the documentation check |
| A web browser | check 11 | E.g. Chromium or Firefox |
To see which tools are missing, run:
You should see (output, don't type it) only tool check done. Each missing: line names a tool to install with sudo apt install (g++ and make come with build-essential).
Checks 3 to 8 change tables (they insert, update and delete rows). So that your real tables in data/ are never touched, they run on a copy of the frozen test tables in a temporary folder. Run these lines once, in the project folder, in the same terminal you will use for the checks:
You should see (output, don't type it) a temporary folder; the letters after tmp. change every time:
PROJECT remembers the project folder, whatever its name; TEST_DIR is the temporary folder. Both exist only in this terminal window: run all the checks in the same window. If you close it or open a new one, run these lines again first.
Run each block below as it is, one line at a time or all at once. Keep each command on its own line: a line ending with & runs in the background, and joining it to other commands with && sends them all to the background too.
You should see the compile commands, and no line containing warning. To count those lines, run:
Why it matters: the Makefile asks for every useful warning (-Wall -Wextra -Wpedantic). Warnings often point at real bugs (an unused result, a wrong type, a missing case), so the project keeps them at zero.
You should see (output, don't type it):
How it works: each tests/NAME.sql is a script of statements, and tests/NAME.expected is the exact output it must produce. tests/run_tests.sh runs every script on a fresh copy of tests/data/ and compares the output with diff. One different character is enough to fail. Together, the 8 scripts cover SELECT, WHERE with every operator and LIKE, INSERT, UPDATE, DELETE, CREATE TABLE, DESCRIBE, COLUMN, apostrophes in values (d'Asti, L'Aquila), and the error messages.
If a test fails, the runner prints the difference: lines starting with - are what was expected, lines starting with + are what the program printed instead.
Adding a test: add statements to a tests/*.sql file (or create a new one), then run tests/run_tests.sh --update to record the output, and read the new .expected file (or git diff tests/) to confirm the output is right before you commit it.
Why tests/data/ and not data/: the tests need tables that never change. If they used your working tables, adding a row on the web page would make the tests fail although nothing is broken. This actually happened during development, which is why the tests got their own frozen copy.
A search with LIKE:
You should see (the first lines; output, don't type it):
A mistake, and the exit code a script returns when a statement fails:
You should see (output, don't type it):
What it shows: errors are reported clearly, the program does not stop, and a script that had a failing statement ends with exit code 1, so other programs can detect it.
First make sure that curl is installed, and that the copy of the tables from "Before you start" is there, in this terminal window:
You should see (output, don't type it):
Start a test server in the background, on a spare port and on the copy of the tables:
You should see (output, don't type it):
If it says "server did not start", the line after it tells you why; see the note at the end of this check.
The terminal prints a line like [1] 12345: that is the background job's number and process ID, and it is normal. When you stop the server at the end of check 6, it prints a line like [1]+ Terminated (in Italian, Terminato): also normal.
Ask it for the page, a file that does not exist, a method it does not support, the list of tables, a statement, and a file outside the page's folder:
You should see (output, don't type it):
200 means OK, 404 "not found" and 405 "method not allowed". The page itself is index.html from the web/ folder; /api/tables and /api/sql are the API the page uses, and they answer in JSON. The last line tries to read a file outside web/ (--path-as-is makes curl send the .. exactly as typed): the server refuses, so nothing outside the page's folder can be read. Leave the server running for checks 5 and 6.
If curl prints 000: it could not connect, so no server is running on port 18080. Usually the server stopped right after starting. The most common reasons:
| Message in "server did not start" | Cause | Fix |
|---|---|---|
bash: /server.log: Permission denied | TEST_DIR is empty: the "Before you start" lines ran in another terminal window | Run them again in this window, then restart this check |
bind to port 18080: Address already in use | Another server (perhaps from an earlier try) still uses the port | Stop it (kill %1, or close that terminal), then restart this check |
Browsers such as Chromium open a connection early and may send nothing on it for a while. This opens such a silent connection, checks that other requests are not blocked by it, then uses it after 6 seconds:
You should see (output, don't type it):
(The time will differ a little; it must be well under one second.)
Why this check exists: an earlier version served one connection at a time and answered a silent connection with an error after 5 seconds. In Chromium, every click then showed "Only GET requests are supported". This check reproduces exactly that situation; the server now gives each connection its own thread.
Send 20 INSERTs at the same moment, and wait until all 20 are answered:
The parentheses run the loop in a subshell, a separate copy of the shell. That does two things: the terminal doesn't announce each of the 20 background requests, and wait waits only for those requests, not for the server (which never ends).
Count the rows, the duplicate IDs, and the different row lengths:
You should see (output, don't type it):
8 original rows + 20 new ones; no ID used twice; every row exactly 250 characters long. The server reads requests in parallel, but runs the SQL one request at a time (a std::mutex), so no two INSERTs can pick the same ID or write into each other.
Stop the test server:
Every table file must have rows of one exact length (the one in SYSTEM_TABLE). This reads your real tables, without changing them:
You should see one length per file (output, don't type it):
(Your list may differ if you created other tables: what matters is one length per file. make check also compares each length with the row length in SYSTEM_TABLE.)
LC_ALL=C makes awk count bytes, as the program does: an accented letter such as ò is 2 bytes.
C++ does not check memory by itself. The compiler can add those checks: AddressSanitizer catches reads and writes outside memory you own, and UndefinedBehaviorSanitizer catches operations C++ leaves undefined (overflows, bad casts). Build a checked copy of the program, then run every test script with it:
You should see (output, don't type it):
The checked program is slower and bigger, so it is only for testing. It stays in build/, which make clean removes.
You should see (output, don't type it):
Doxygen warns about every function, parameter or return value without a comment, so 0 warnings means the whole code is documented. (It needs Doxygen; see INSTALLATION.md.)
The checks above use your working folder. This one checks that the project also works for someone who downloads it for the first time:
You should see 0, then the same 8 PASS lines as in check 2.
Why this check exists: a fresh copy has no build/ folder and nothing left over from earlier work. That is exactly how a bug in make doxygen was found: it worked on the developer's machine, where build/ already existed, and failed on a clean one.
Start the server on the copy of the tables, then open http://localhost:18080 in a browser (tested with Chromium and Firefox):
| Step | Expected |
|---|---|
| Open the page | The tables on the left; EMPLOY (or the last table you used) with its rows |
| Click COMPANY on the left | The companies (8, plus the 20 test firms if you ran check 6), each row with Edit and Delete |
| Click SYSTEM_TABLE | Its rows, with no + New row and no Edit/Delete: the catalogue is read-only |
Click EMPLOY; in Search in keep NOME, type fran, click Search | The rows whose NOME contains "fran" (Francesco, Franco) |
Click + New row, fill every field, with an apostrophe in one (D'Annunzio), click Add row | A "1 row inserted" message; the form closes and the new row is the last one, apostrophe included. The SQL above the rows shows 'D'Annunzio' as typed |
| On that row, click Edit, change a value, press Escape | The row is unchanged, and the SQL line above the rows still shows the INSERT: nothing was sent |
Click Edit again, change a value to one with a space (Reggio Emilia), press Enter | "1 row updated", the new value shown |
| Click Delete on it, then Cancel in the dialog | Nothing is deleted |
| Click Delete again, then the red Delete | "1 row deleted", the row is gone |
In the SQL console, type SELECT * FROM COMPANY WHERE TOWN = Roma and, on a new line, SELECT FOO FROM COMPANY; press Ctrl+Enter | Two results: the rows with a green check, and the error in red |
Click + New table; name cars; first column model, Text, 20; click + Add column: year, Number, 4 | The SQL under the columns reads CREATE TABLE CARS (ID NUMBER(10), MODEL CHAR(20), YEAR NUMBER(4)) while you type |
| Click Create table | "table CARS created with 3 columns (row length 34)"; CARS appears on the left, open and empty, with + New row |
Click + New table again, name cars, any column, Create table | The form stays open and shows "table CARS already exists" in red; click Cancel |
| Click ⚙ Settings, choose Extra large, click Done, then reload the page | The whole page (text, buttons, spacing) is bigger, and still is after the reload; set it back to Normal |
| Switch your computer to a dark theme (optional) | The page follows it, with dark colours |
Above the rows, the page always shows the SQL the last action sent and the server's answer: reading it is part of the check. Stop the server with Ctrl+C.
After each push to main, GitHub rebuilds the documentation site. In the repository's Actions tab, the latest "Doxygen documentation" run must have a green check. From a terminal, without logging in (the repository is public):
You should see (output, don't type it):
"conclusion": "failure" means the run failed: open it in the Actions tab to see why."status": "in_progress" or "queued" means it is still running: try again in a few minutes.API rate limit exceeded.Remove the temporary copy and everything in it:
| After changing... | Run checks |
|---|---|
| Any code | 1, 2 (and 8 for anything that handles memory or text) |
Server.cpp or Output.cpp | 1, 2, 4, 5, 6, 11 |
The tables in data/ | 7 |
| Comments in the code | 9 |
| The Makefile, or before a release | all of them |
| Bug | Found by | Fixed |
|---|---|---|
The last character of every column was lost (13743 shown as 1374) | Comparing real output with the data (now check 2) | Columns read from FROM to TO inclusive |
| Chromium showed "Only GET requests are supported" on every click | A browser test, then reproduced by check 5 | One thread per connection; silent connections closed without a reply |
| The tests failed after a row was added on the web page | Check 2 | Tests use their own frozen tables in tests/data/ |
make doxygen failed on a clean machine (GitHub) | Check 12, then check 10 | make doxygen creates its output folder first |
A plain wait in a test script waited forever for the server | Writing check 6 of this guide | Run the 20 requests in a subshell, whose wait ignores the server |
On a fresh Ubuntu: curl missing, and an output box (0) typed as a command | A user following this guide | "What you need" section; output boxes labelled "You should see" |
On a fresh Ubuntu 26.04: curl still missing at check 4 | A user following this guide | A one-line tool check in "What you need", and a curl check at the start of check 4 |
The last one is a reminder that test scripts are code too, and need testing as well.