Microservices

Introduction

In this project, you will implement, optimize, and evaluate a microservice-structured application. The application is a simple restaurant management system. It can store and provide details about restaurants.

Our application consists of the following microservices:

• Detail: provides and modifies restaurant details
• Database: stores and retrieves data with persistence
• Cache: (unimplemented): caches data

These services then communicate with each other via well-defined APIs. For this project, we will use gRPC to define these APIs.

One could imagine implementing the restaurant management system as a monolithic application – meaning all functionality is contained within a single program. However, adopting a microservices architecture allows for more flexibility and scalability. By splitting the application into smaller services, individual functionality can be developed independently and potentially in different languages. It also allows for individual services to be scaled as needed.

Acknowledgements

Much of the setup for this project is taken (with permission) from Simon Peter’s UW CSE 453.

Setup

Building the project produces a binary microservices that can run any of the services. For example, microservices detail will run the detail service and microservices database in-memory will run the database service using in-memory storage. Although you could potentially run all services in separate terminals, it is recommended to follow the instructions below.

There are two main ways to run this project: locally or using Kubernetes.

wrk2 setup

We can test our application for correctness using curl. However, if it’s the kind of thing you find fun, we’ve provided a stub website, in ./static, as well that will let you use the application from the browser. The web implementation is incomplete.

Eventually, though, we want to stress-test our implementation. Although in projects 0 and 1 we implemented the load generator ourselves, for this project the starter code includes wrk2, an HTTP load generator.

wrk2 uses a submodule for LuaJIT, one of its dependencies. Make sure you check out the submodule by running git submodule update --init --recursive.

To compile wrk2 (if not using Debian/Ubuntu, you’ll need the equivalent packages on your distro):

cd wrk2
sudo apt install luarocks libssl-dev libz-dev make
sudo luarocks install luasocket
make

We’ve also provided a Dockerfile, at wrk2/Dockerfile, to help build wrk2. This should help those of you who need to run on macOS. After compiling (docker build -t wrk2:1 .), you’ll want to run the container with --network host, so that you can send traffic from wrk2 as if it was on your host network.

Local Setup

When developing locally, it is useful to not have to go through the process of building and deploying the application every time a change is made. To facilitate this, we provide a script util/run that will start all services in separate tmux panes. It also provides an additional pane with a shell to run curl commands to interact with the services.

To run the application locally, make sure the microservices binary is built using cargo build --release. Then, run ./util/run to start the application. While in the tmux session, type Ctrl+b followed by :kill-session to end the session.

To learn more about how to use tmux, see here.

Kubernetes Setup

Kubernetes is an automated containerized application management platform used for deploying, scaling, and managing applications using containers. It makes it easier to implement common management features like configuring containers (by making the config declarative), providing resilience by automatically restarting containers when they crash, and scalability by starting new container instances in response to pre-defined criteria. Docker runs containers; Kubernetes (also called k8s, to save those precious keystrokes) manages those containers. Part of this project is configuring your application on Kubernetes.

A note on vocab: You will see the term “pod” appear below. For the purpose of this project, the Kubernetes term “pod” and the term “container” we used in class are interchangeable. You can read more about the exact difference here; basically, in some deployments, a pod can consist of multiple containers. A Kubernetes “deployment” is a set of pods to start.

To test using Kubernetes locally, the easiest way is to use Minikube, which is a lighter-weight version of the same functionality designed for testing environments. Follow Minikube’s instructions to set it up. This project also uses Helm, a way of installing applications in Kubernetes clusters. You should install Helm from its instructions.

Minikube running

• Make sure Minikube is running using minikube start --nodes 2. This will start a local cluster with two nodes (both of them on your local machine), which is the most similar setup to the class VMs.

This raises a question: which pods should run on which nodes? The way Kubernetes manages this is with node labels. The idea is that your manifest can declare that a pod should run on a node with some property. In our case, we’re just concerned with locality, so the node labels we give you will just be name=1675-node1 and name=1675-node2. In your Minikube cluster, you should assign these labels to your two virtual nodes. You can optionally write a “node selector” in your pod declaration as described in the linked docs to control which pods run on which node.

• Build the container image using minikube image build -t cs1675/microservices:v1 . (You can change the image name, but it should match manifests/values.yaml)
• Ensure the image shows up in minikube with minikube image ls.
• Deploy the application using kubernetes using helm install 1675-microservices-app ./manifests --wait --namespace 1675-microservices --create-namespace.
• Test your application. The frontend microservice should be visible outside the Kubernetes cluster. Ensure this is the case with minikube service frontend.
• Later, delete the deployment using helm delete -n 1675-microservices 1675-microservices-app --wait. You can delete the namespace we created for our application with kubectl delete namespace 1675-microservices.

Useful kubectl commands: - kubectl get deployments to see running deployments. - kubectl get pods to see running pods. - kubectl exec -it <pod-name> -- /bin/bash to enter a pod.

Class VMs

On the class VMs, we won’t use Minikube, since we have multiple VMs to work with. Instead, we will run a cluster with kubeadm. To run your application in this environment, we override various values in the Helm chart, including the name of the image and the namespace to prevent clashes between various students’ code.

Requirements

This project has both benchmarking (🔎) and implementation (🛠) requirements.

Summary of Deliverables

We will discuss the following aspects of your report:

• Implementation correctness. (1 point)
• Instrumentation to measure the difference between in-memory and file-based database (and between other configurations). (1 point)
• An optimized implementation of the file-based database. (1 point)
• Two cache designs. (2 points)

🔎 Part 1: Using the Load Generator

Request types to consider

The two types of requests are:

1. Retrieve Restaurant Details: GET /get-detail/

Retrieves restaurant details.

Request Format (JSON):

{
    "restaurant_name": "..."
}

2. Update Restaurant Details: POST /post-detail/

Updates or creates restaurant details.

Request Format (JSON):

{
    "restaurant_name": "...",
    "location": "...",
    "style": "...",
    "capacity": ...
}

Loading script

To avoid a bunch of unhelpful 404s, there is sample data in samples/detail_samples.csv and a loading script to POST this data in util/load.py. You should run this script before running wrk2.

Zipfian access pattern

Our workload, specified in workloads/zipf-mixed.lua, follows a Zipf distribution. Recall that we also used a Zipf distribution for project 2’s workload. In particular, given a dataset size N and scaling exponent α ≥ 0, the probability of the i^th (such that 1 ≤ i ≤ N) entry occurring is

P(X=i | \alpha, N) = \frac{1}{i ^ \alpha} \cdot \frac{1}{\sum_{j =1}^n{1/j^\alpha}}

For this project, wrk2 will randomly sample requests from a Zipfian distribution with α = 1.5 and N = 100.

Using wrk2

Sample invocation:

$ wrk2/wrk -D const -t 1 -c 1 -d30s -s workloads/zipf-mixed.lua http://<ip>:8080 -R 5 -P > wrk.txt

🔎 Part 2: Instrument In-Memory vs. File-Based Storage Performance

When the application starts, it can be configured to use different database implementations. The database service can be run using two storage backends: - In-Memory (microservices database in-memory) - File-Based (microservices database file /tmp/microservices.db)

These implementations can be selected by passing the appropriate argument to the microservices database command. If running locally, edit the util/run script to start the database service with the desired storage backend. If running in Kubernetes, edit the manifests/values.yaml file to change the storage backend.

Your report should both report the difference in end-to-end performance and explain this difference with quantitative evidence. In previous projects, we used perf to generate flamegraphs for this purpose. However, running perf in Kubernetes environments is usually not possible in real deployments, since it requires disabling some kernel protections. As a result, you will have to implement a different method of gathering instrumentation data.

Hint: A simple implementation might modify the application to add some logging. A more complicated implementation could, for example, modify the Helm chart to deploy as a dependency an eBPF monitoring tool such as Tetragon. Whichever way you choose, the grading server will save any .txt file you produce in the log output you can download.

🛠 Part 3: Optimize Persistent Storage

Optimize the database service to improve performance. Your report should explain your design and include a quantitative evaluation of its effectiveness relative to the FileDB implementation in terms of both end-to-end performance and the instrumentation you implemented in part 2.

For reference, the FileDB implementation is defined in src/database/filedb.rs. src/main.rs defines an option FileOpt, which is left for you to implement.

Hint: Think about how you can reduce the number of system calls the implementation makes.

Hint 2: Remember that you must still maintain the application’s correctness. What happens if the database crashes? You might want to write a test that covers this (e.g., run kubectl -n 1675-microservices delete pod -l name=database halfway through your test and let the Deployment we created restart it), but this is not required.

🛠 Part 4: Implement and Deploy Cache Service

Even an optimized file-based implementation will have some performance overhead relative to the in-memory configuration. To optimize this further, design, implement, and deploy a cache microservice. See src/main.rs, src/detail/mod.rs, src/cache/mod.rs, build.rs, and src/proto/mod.rs for which parts of the starter code to modify. src/main.rs takes an argument that specifies the size of the cache. Please respect this parameter. Remember that the workload is Zipf-distributed, like in project 2.

You should implement two different cache designs which differ in their access (e.g., look-aside vs look-through) and/or eviction (e.g., FIFO vs LRU) policies. Your report should explain your two designs and include a quantitative evaluation of their effectiveness, comparing (a) your FileOpt database implementation with each of your cache designs, (b) your FileOpt database implementation without your caches (i.e., Part 3) and (c) the in-memory database implementation (i.e., Part 2). As before, consider both end-to-end performance as well as your instrumentation from part 2.