This weekend I read Google’s F1 paper. It was a satisfying read because it builds over various infrastructure papers published by Google before. This years VLDB conference was full of HTAP database advancements. I am also actively working in this field. Feel free to connect if you want to know more.

P.S - the following is the compilation of my reading, lots of plagiarism but I wanted to present key things in front of you. I have citied the main sources in the end.

Share Designing Distributed Systems

Google F1 SQL Engine

Google’s F1 powers AdWords, hence it is a critical system because Advertising generates 88%+ of Google’s revenue.

Key Ideas -

The design of query system is very simplistic.
F1 lightning focuses on maximizing the parallel request to the remote spanner storage. This results in reduction on network roundtrips and lower contention of a single disk in terms of disk seeks.

F1’s addition to Spanner

A full SQL layer with distributed SQL query execution; including joining of data from external sources. This is largely the bridge between the data storage and consistency functionality of Spanner, and the AdWords application developers who are familiar with legacy MySQL system.
Transactionally consistent local + global secondary indexes - this is for AdWords application only.
Async schema changes: an external system checks and maintains the system in face of continuous schema changes - this guards against data corruption and maintains availability.
Optimistic Transactions: this is built on top of Spanner’s capabilities, effectively leveraging a CAS implementation combining Spanner’s lock-free snapshot reads and consistent transactions.
Automatic history recording and publishing - another feature for the users, includes a creative and effective cache application.

Key design choices - F1:

The data schema is explicit in the sense it exposes data clustering close to the application programmer. This is a shared concept with F1. Concretely, F1 stores data with a hierarchical structure (outlined later). This layout is optimised with Spanner storage in mind, and the result is reduction of RPC trips.
F1 heavily uses batching, parallel processing, and async reads
Subscribe now

Architecture overview:

F1 System consists of -

Client
Load Balancers
the most commonly communicated F1 Server
F1 Master and
F1 Slaves.

F1 Servers are mostly stateless except over the duration of a transaction, during which the F1 Server is a client of Spanner, and holds pessimistic write leases/locks.

F1 Master and Slaves are responsible during distributed SQL query executions. Masters maintain pool membership over the Slave pool. Slave nodes are responsible for processing the distributed queries on behalf of the coordinating F1 Server that initialized the query.

Scaling the F1 system is trivial - simply add more nodes, which is done quickly. Adding new Spanner nodes results in data re-distribution (copying of Spanner directory files).

Due to the nature of remote data storage, minimum commit latency is relatively high 50-150ms compared to MySQL. The F1 ORM client library exposes this effect and forces the application developer to minimize serial tasks.

Query Processing -

The F1 SQL query processing system has the following key properties which we will elaborate on in this section:

Queries are executed either as low-latency centrally executed queries or distributed queries with high parallelism.
All data is remote and batching is used heavily to mitigate network latency.
All input data and internal data is arbitrarily partitioned and has few useful ordering properties.
Queries use many hash-based repartitioning steps.
Individual query plan operators are designed to stream data to later operators as soon as possible, maximizing pipelining in query plans.
Hierarchically clustered tables have optimized access methods.
Query data can be consumed in parallel.
Protocol Buffer-valued columns provide first-class support for structured data types.
Spanner’s snapshot consistency model provides globally consistent results

If you want to read more about F1 lightning and its amazing capabilities, please checkout -

https://research.google/pubs/pub41344/

https://warrenqi.github.io/2020/03/29/google-f1-2013.html

Please feel free to reach out to me if you need help with building large scale distributed systems or data engineering solutions at your organisation, I would be more than happy to help.

If you are liking my articles and want me to write more then feel free to buy me a coffee! 😁

I hope that you enjoyed this read, if so then please share this article with your friends, let’s build a solid community. I will be back next week with another well thought/researched article delivered straight in your inbox. See you!

You can connect with me on Twitter.

Designing Distributed Systems

Google and its distributed SQL engine