Articles 2026-11-01 Open Access

Effect of Index Design on Query Latency, Storage Overhead, and Write Throughput in SQLite: An Empirical Benchmark

PDF

Abstract

Secondary indexes are the main tool for speeding up relational queries, but every index also costs storage and slows down writes. This study measures that trade-off in SQLite 3.45.1 using a synthetic orders table of 10,000, 100,000, and 1,000,000 rows. Three index configurations were compared (no secondary index, single-column indexes, and composite or covering indexes) across five query types: point lookup, range aggregation, multi-predicate filtering, a join with grouping, and a top-k query. Without secondary indexes every query was a full table scan, and latency grew roughly in proportion to table size; at one million rows the point lookup took 46.1 ms. Single-column indexes cut this to 0.04 ms, a speedup of about 1,155 times, yet they improved the 30-day range aggregation by only 1.3 times. Composite and covering indexes improved the same query by 26.6 times and the multi-predicate query by 127 times. The join query gained 3.6 to 4.1 times under both indexed configurations. These gains came at a price: at one million rows the database grew from 34.0 MB to 71.9 MB and 95.7 MB, and sustained insert throughput fell from about 673,000 rows/s to 34,000 and 17,000 rows/s. The results show that index benefit depends on how well an index matches a query's predicates and projected columns, not merely on whether an index exists, and that write-heavy workloads need to weigh these costs carefully.

References

Bayer, R., & McCreight, E. (1972). Organization and maintenance of large ordered indexes. Acta Informatica, 1(3), 173–189. https://doi.org/10.1007/BF00288683

Chaudhuri, S., & Narasayya, V. R. (1997). An efficient cost-driven index selection tool for Microsoft SQL Server. In Proceedings of the 23rd International Conference on Very Large Data Bases (pp. 146–155). Morgan Kaufmann.

Comer, D. (1979). The ubiquitous B-tree. ACM Computing Surveys, 11(2), 121–137. https://doi.org/10.1145/356770.356776

Gaffney, K. P., Prammer, M., Brasfield, L., Hipp, D. R., Kennedy, D., & Patel, J. M. (2022). SQLite: Past, present, and future. Proceedings of the VLDB Endowment, 15(12), 3535–3547. https://doi.org/10.14778/3554821.3554842

Graefe, G. (2011). Modern B-tree techniques. Foundations and Trends in Databases, 3(4), 203–402. https://doi.org/10.1561/1900000028

Jain, R. (1991). The art of computer systems performance analysis: Techniques for experimental design, measurement, simulation, and modeling. John Wiley & Sons.

Kraska, T., Beutel, A., Chi, E. H., Dean, J., & Polyzotis, N. (2018). The case for learned index structures. In Proceedings of the 2018 International Conference on Management of Data (pp. 489–504). ACM. https://doi.org/10.1145/3183713.3196909

Selinger, P. G., Astrahan, M. M., Chamberlin, D. D., Lorie, R. A., & Price, T. G. (1979). Access path selection in a relational database management system. In Proceedings of the 1979 ACM SIGMOD International Conference on Management of Data (pp. 23–34). ACM. https://doi.org/10.1145/582095.582099

SQLite Consortium. (n.d.). The SQLite query optimizer overview. https://www.sqlite.org/optoverview.html