Newark Civic Pulse
Evidence with a date and a source.
Last successful source fetch: 2026-11-10T19:00:20.078116+00:00.
8 current vector embeddings, 1536 dimensions. AI answers are configured.
The methods behind the civic briefing
How Ask Newark works.
From an official record to a searchable passage to an answer you can check.
This is an independent student research project. The methods and results below are AI-authored and AI-reviewed, with primary sources linked. Human verification and peer feedback remain separate, unfinished work.
The current collection
7 source records.
8 searchable passages.
A small snapshot of recent Newark council meetings and selected agenda items. It does not cover every meeting, decision, city service or budget figure.
Try Ask NewarkKeep the research and the app together.
Open the combined individual record for the portfolio, original Plan and source/comparison/decision index.
01 / Get the source right.
Extract → transform → loadThe research question is small: can a resident ask about imported council records, inspect the relevant evidence and receive an answer that stays within it? The corpus comes from official source records displayed by this site. Account forms, navigation and design-preview copy are excluded from retrieval.
Read the public feed.
The Newark City Clerk links to the city's Legistar agenda service. The importer requests a bounded set of public meeting and agenda metadata from that official API.
An anonymous read succeeded for this source. Legistar's API documentation says token rules vary by client; this observation does not promise permanent access or authorize reuse of unrelated attachments and assets.
Keep meaning and provenance.
Validate record IDs and dates, normalize text, preserve the local meeting timezone, and retain the original URL, fetch time, source update time and content hash.
Build readable meeting or agenda-item text, then split only when needed. An agenda's publication status does not establish that an item passed, and a listed schedule does not prove a meeting occurred.
Save a consistent version.
Commit sources and their dependent chunks in one SQLite transaction. Unchanged imports reuse stable identities; changed sources replace their old searchable passages together.
Refresh compatible embeddings for changed chunks. Failed or incomplete imports preserve the last successful snapshot. A source change during answer preparation returns a retryable conflict instead of mixing revisions.
Executed import: the November 10 research snapshot contains five recent meeting records and two selected agenda items: seven sources and eight chunks. After source-link repairs, a repeat import reported all seven unchanged. These are observed bounded-import results, not a complete Newark calendar.
02 / Make useful pieces.
A chunk is a passage with its context attached.Current method
Keep a short civic record together.
One short meeting record becomes one passage containing its date, scheduled time, place and source. A longer agenda item is split at paragraph or word boundaries while retaining its heading and source version.
The present cap is 1,600 Unicode characters, with zero overlap. Characters are not model tokens; this boundary experiment does not establish an optimal tokenizer setting.
For the November 5, 2026 record, preserving 12:30 PM, America/New_York, Council Chamber together makes its schedule easier to inspect. The date must remain attached so a past record does not appear to describe an upcoming meeting.
03 / Search, then answer.
Different rankings answer different questions.FTS5 + BM25
SQLite FTS5 searches the imported text. BM25 ranks word matches using factors such as term frequency and document length. This is useful for dates, ordinance numbers and exact names.
Exact words can still miss a differently phrased question. Retrieval is evaluated separately from the generated answer.
Embeddings + cosine distance
OpenAI's text-embedding-3-small represents each passage and question as a numeric vector. The evaluated index uses 1,536 dimensions. sqlite-vec retrieves nearby vectors using cosine distance.
Nearby text can be irrelevant or incomplete. Similarity does not establish truth, correct dates or support for every answer claim.
Reciprocal rank fusion
The hybrid path combines lexical and vector rankings by adding 1 / (60 + rank) for each appearance, then selects up to five passages.
RRF combines ranks without treating BM25 scores and vector distances as the same scale. Our small evaluation did not show a source-coverage advantage over keyword search.
Generation
An answer must point back to evidence.
The bounded retrieved passages are supplied to gpt-5-mini. The model is instructed to use that evidence, preserve dates and uncertainty, ignore instructions within source text, and report insufficient evidence when necessary.
The application accepts only references to retrieved current chunk IDs and adds the source links itself. Citation validity is a useful mechanical check; claim-level support still needs inspection.
Inspect passages in Ask Newark ↗04 / What informed the decisions.
Five primary research papersThese papers inform design choices. Their reported benchmark results do not become results for Newark Civic Pulse.
Lewis et al. (2020) · Retrieval-Augmented Generation
Finding used: external retrieved passages can supply evidence to a generator. Decision: retrieve current civic chunks before generating an answer and retain the trace. Limit: this API-based application does not reproduce the paper's trained retriever–generator system. Sections 2–2.2 informed the review.
Karpukhin et al. (2020) · Dense Passage Retrieval
Finding used: dense passage/query representations support retrieval, which can be assessed separately from generation. Decision: compare required-source coverage at a fixed top-five budget against the lexical baseline. Limit: its benchmark gains do not establish gains for our corpus or OpenAI embeddings. Sections 3 and 5 informed the review.
Wang et al. (2024) · RAG pipeline and chunking choices
Finding used: chunk boundaries, metadata and retrieval combinations introduce quality/efficiency tradeoffs. Decision: retain meaningful civic-record boundaries and compare paragraph splitting with fixed windows. Limit: the paper's models, sizes and preferred pipeline require local evidence before adoption. Sections 3.2, 3.4 and 5.1 informed the review.
Es et al. (2023) · RAGAS
Finding used: faithfulness, answer relevance and context relevance are different evaluation dimensions. Decision: review retrieved evidence separately from whether the answer's claims follow from it. Limit: no RAGAS score or automated-judge result is claimed here. Section 3 informed the review.
Liu et al. (2023) · Lost in the Middle
Finding used: evidence placement and distracting long context affected the models/tasks studied. Decision: supply a small ranked context rather than the whole site. Limit: a position effect has not been measured for this application's current model. Section 2.3 informed the review.
05 / What was actually checked.
Frozen local evaluation · November 10, 2026Five questions and expected sources were fixed before five real question-embedding requests and five real model-answer requests. The saved evaluation used seven sources/eight chunks, hybrid context, text-embedding-3-small and gpt-5-mini. There were no automatic retries.
| Question and expected evidence | Keyword | Vector | Hybrid | Answer evidence review |
|---|---|---|---|---|
| Q1 · Exact term November 5 meeting: listed time and location. | 1/1 sources | 1/1 | 1/1 | Supported: 12:30 PM, Council Chamber. Prefer “the record lists” to wording that sounds upcoming. |
| Q2 · Paraphrase The same session's place and local clock time. | 1/1 | 1/1 | 1/1 | Source fields match. “Residents were directed” is stronger than the stored location field; use the record's limited wording. |
| Q3 · Relationship Ordinance 26-0816's agenda item plus its meeting's start time. | 2/2 | 1/2 | 2/2 | Supported by both records. Vector-only retrieval missed one required source; hybrid restored coverage. |
| Q4 · Comparison November 5 and October 21 scheduled local times. | 2/2 | 2/2 | 2/2 | Supported: 12:30 PM versus 6:30 PM, retaining the dates and timezone. |
| Q5 · Unanswerable Total approved 2026 Newark budget in dollars. | Not applicable | Not applicable | Not applicable | Appropriately reported insufficient evidence, with no invented amount or citation. |
Result: keyword and hybrid each found every required source set on 4/4 answerable questions; vector did so on 3/4. This is source coverage, not precision or proof of general superiority. Generated-answer quality was reviewed using hybrid context; it was not compared across all three retrievers.
Read the complete answer/evidence review and frozen machine-readable receipt for the exact questions, ranked IDs and model outputs.
06 / Keep the limits visible.
Evidence supports a bounded claim.Listed facts, with dates.
Imported meeting dates, scheduled times, locations and selected agenda listings can support limited questions. The collection cannot establish the next city meeting, a total budget, a vote or final passage without the relevant evidence.
A broader, human-reviewed sample.
More source families, questions, tokenizer-aware boundaries, citation support and useful peer feedback need checking. Dollar spend and latency were not measured in the frozen evaluation. A small success is not a general accuracy or security certification.
Keep authorship and review honest.
The portfolio foundation, learner claim verification, real peer feedback and the resulting change/retest remain pending. Private career reflection belongs in Canvas. This research page does not submit the assignment or complete the individual PowerPoint.
Follow the evidence yourself
Ask a small question.
Check its sources.
Start with a dated question about the imported council records. Inspect the returned passages and distinguish a scheduled event from a verified outcome.
Try the bot
Open the combined individual record
Research and evaluation snapshot: November 10, 2026. Collection counts above reflect the current app database.