Today was the second day of my schedule, and it had three blocks on the agenda:
- 2.5 hours of Python concepts and exercises
- 1 hour of system design
- 1 hour of AI tool investigation
Setbacks
Despite hoping that yesterday’s slow start was a one-off, procrastination got the better of me again, and I had a late start to the first time block on the calendar. As a result, I didn’t get through everything I had planned, and the AI tooling block ended up being pushed to later in the week.
Python Exercises
I split my Python time into two parts: an hour in the morning and the remaining hour and a half later in the day.
I continued working through the Python track on Exercism, finishing the conditionals section and starting on strings. I got stuck for a while trying to satisfy the various test cases for the Pig Latin exercise. It felt very tedious, but it was good practice with Python’s substring syntax. After that, I moved on to Little Sister’s Vocabulary to begin the string exercises.
Along the way, I noticed that one of my completed exercises had a yellow circle indicator next to the green one that showed it had passed. Viewing my solution in the browser revealed Pylint feedback and AI-generated suggestions for improvements, which was a helpful surprise.
System Design
I tend to prefer video tutorials over long-form articles. A while ago, I came across the Hello Interview YouTube channel and watched a couple of their videos. I liked what I saw, so it felt like a good place to start. They also have a website with a lot of written content to go along with the videos.
I started with a video on database sharding that I had bookmarked a while back. Sharding is the practice of splitting data across multiple servers, with each server (or shard) holding a subset of the data.
Choosing a Shard Key
A good shard key has three properties:
- High cardinality: the field has lots of unique values.
- Even distribution: data is spread roughly evenly across the shards.
- Alignment with queries: common queries shouldn’t need to fetch data from more than one shard.
The video covered hash-based sharding using consistent hashing around a hash ring, and presented it as the preferred sharding strategy.
Hot Spots
A hot spot occurs when one or more shards grow much faster than the others. The example given was a system where posts are sharded by user ID, and one of those users is a celebrity with a very high volume of activity.
One solution is to use a compound shard key to distribute that user’s data further. Another is to create a dedicated “celebrity shard”, which requires a directory-based lookup before writes to determine whether a particular user belongs on it. It was noted that this is a fairly rare problem in practice.
Cross-Shard Operations
Sticking with the same example, if posts are sharded by user ID but someone requests the most popular posts across all users, the query has to hit every shard. This problem can’t be eliminated, but it can be mitigated.
One mitigation is to cache the results of cross-shard operations. Another is denormalization, where common data is written to multiple shards so that reads only need to query one. The trade-off is that writes now have to go to multiple shards, making them slower in exchange for faster reads.
Consistency
When related data lives on different shards, updates can no longer be made in a single atomic transaction. Two-phase commits are one option, but they’re not ideal, since they lock resources across multiple shards and can fail if any one of them becomes unresponsive.
The preferred approach is the saga pattern, which chains together a series of smaller steps. It requires more overhead, since each step needs a compensating action to undo it in the event that a later step fails.
When Not to Shard
One of the more useful takeaways was that recognizing when sharding isn’t necessary is just as important in a system design interview. Peak read and write throughput are the most significant factors in that decision, and a single database running on large hardware can often handle extremely large datasets, removing the need for sharding in many cases.
Takeaways
The Python exercises feel tedious because the programming is so basic, but I understand the value of putting in the reps to become fluent in the fundamentals, and I can feel it helping. Once I’ve worked through the Exercism track, I’m looking forward to getting familiar with more of the frameworks and tools in the Python community.
The system design content, on the other hand, was incredibly interesting. I haven’t had the opportunity to work directly on systems with sharded databases, and it’s felt like a technical blind spot for me for many years. Much less so after today, and it’s something I’d love the chance to work on in the future.
So far, I’ve been treating the time blocks on my calendar as flexible within each day. I want to get better at sticking to them as scheduled, and once this first week is over, I’ll look at adjusting the schedule with Claude based on how it’s gone.
