Use PostGIS for bus positions

README.md

@@ -1,9 +1,9 @@
 # Jokullbase
 
 How to track how late buses are:
- - [ ] Download bus route data
+ - [x] Download bus route data
- - [ ] Insert raw bus positions into timescaledb/postGIS table?
+ - [x] Insert raw bus positions into timescaledb/postGIS table?
-   - [ ] "Fan-out" JSON responses into one DB row per datapoint.
+   - [x] "Fan-out" JSON responses into one DB row per datapoint.
  - [ ] Download routes.
    - There is an endpoint called Timetable, which takes a route number
      (along with day of week etc).
@@ -14,6 +14,8 @@ How to track how late buses are:
 	 - This information has upcoming arrivals, included estimated
        arrival time, along with the current position of the bus.
    - Record stop info every few minutes, notably the arrivals.
+   - This will be annoying to do only in DB since we have a trigger
+     for bus positions.
  - [ ] Analyze actual arrival vs what the stop endpoint said about the
        arrival.
    - Use PostGIS to compute "bus arrived" event: When bus is within X
@@ -23,3 +25,16 @@ How to track how late buses are:
    - We can then discard the raw bus position data, as it's not needed
      to store it: delete every raw data point between the last arrival
      and the newly computed one.
+
+Arrival computation:
+ - Can use edge functions for this?
+ - Function takes a bus route number, start time, end time.
+ - Cron in DB calls the edge functions, using state as function inputs.
+   - state = last interval end time, so we can properly calculate time interval to check.
+   - or rather use last ID in the raw table, so we don't skip entries.
+ - Query DB to get raw bus positions in the interval.
+ - Query DB to get all stops on the route (during the interval? some stops not in use sometimes)
+ - For each bus ID, find the raw position where its lat,lon are X meters away from stop lat,lons.
+ - These are arrival times for this bus ID, at each stop.
+ - Insert into arrivals.
+ - After all arrivals computed, drop all raw positions for this bus from DB.

supabase/migrations/20240409190001_initial.sql

@@ -3,6 +3,7 @@ CREATE EXTENSION IF NOT EXISTS "plv8" SCHEMA pg_catalog;
 CREATE EXTENSION IF NOT EXISTS "pg_cron" SCHEMA pg_catalog;
 CREATE EXTENSION IF NOT EXISTS "pg_net" SCHEMA extensions;
 CREATE EXTENSION IF NOT EXISTS timescaledb CASCADE;
+CREATE EXTENSION IF NOT EXISTS postgis;
 
 CREATE TABLE IF NOT EXISTS raw_bus_positions (
   id BIGINT GENERATED ALWAYS AS IDENTITY,
@@ -14,8 +15,7 @@ CREATE TABLE IF NOT EXISTS raw_bus_positions (
   headsign text,
   tag text,
   direction int,
-  lat decimal,
+  coords geometry(Point, 4326)
-  lon decimal
 );
 
 -- TimescaleDB Hypertable creation

supabase/migrations/20240410070316_scheduled-jobs.sql

@@ -35,12 +35,12 @@ CREATE FUNCTION copy_to_raw_table() RETURNS trigger LANGUAGE plpgsql AS $$
 	     )
     )
     INSERT INTO raw_bus_positions
-	-- This needs to be in the same order as the jsonb_to_recordset call to work properly.
+	(response_status, measured_at, bus_id, trip_id, route_number, headsign, tag, direction, coords)
-	(response_status, measured_at, bus_id, trip_id, route_number, headsign, tag, direction, lat, lon)
       SELECT
 	NEW.status_code,
 	(NEW.content::jsonb->'data'->'BusLocationByRoute'->>'lastUpdate')::timestamptz,
-	mr.*
+	mr."busId", mr."tripId", mr."routeNr", mr.headsign, mr.tag, mr.direction,
+	ST_SetSRID(ST_MakePoint(mr.lat, mr.lng), 4326) -- PostGIS coordinates
       FROM mapped_response_rows mr
       JOIN raw_bus_position_requests raw_request ON raw_request.request_id = NEW.id
     WHERE