Peer Review
1 Importance of Peer Review
Being able to read others’ code. In fact, you rarely get to write a software system from scratch; most of your time will be spent maintaining and extending existing code.
Being able to give feedback on others’ work. This is a common practice called peer review.
In addition to helping you prepare for the future, engaging in peer review will also help you see the world through the course staff’s eyes. After all, whenever you turn in a homework, we have to read and make sense of your solutions. If you find someone else’s code difficult to read, most likely so did a course staff member.One of the hidden benefits of being a TA is that you get good at reading code! Likewise if it’s your code that’s hard to read. We hope that by struggling to read others’ solutions, you will internalize some lessons about how to make your own code readable. You will realize, for instance, that the “style” guides aren’t there to make your life difficult, but rather to make your code legible. It’ll become clear the first time you read someone who didn’t follow them!
2 What We Will Do
To give you a feel for this, while accommodating the time constraints of the semester, we will have you review problems that are familiar but programs that might not be. That is, you will review your classmates’ solutions as well as receive feedback on your own.
We will use a system called Peerceptiv to conduct peer review. We are purposely not linking to it here because you should not go to it directly. Instead, you should go to the course’s Canvas page and access it through the link on that page (that’s how Peerceptiv associates you with your Brown identity; otherwise you won’t get credit for your work).
For uploading your homework. This is before the start of the PR period listed on the course calendar. You must finish uploading before the assignment period begins, so that all the assignments are available to distribute to everyone else when it begins.
For reviewing others’ work. This is the end of the PR period listed on the course calendar. You have to get your reviews done by this time.
For reflecting on others’ reviews. Spilling over past the end of the PR period, we ask you to read others’ comments and write a reflection. This may overlap with the next assignment.
All three deadlines are firm: there are no extensions (because it is impossible to give extensions without disrupting everyone else, and the software works best when all the assignments are in place to distribute at the deadline).
2.1 Uploading
The code is anonymized. It should have been already, but double-check and, if it hasn’t been, make sure it is now.
You upload in one of two formats: .txt or .pdf. That is, download the file and rename the file from .arr to .txt. (If you wish you can further convert it to PDF.) Do not upload the file with the name .arr! Peerceptiv does not know how to process such a file, so the people who receive it to review it will not see anything. They will complain/ask for help, but there is no time to fix things.
You’re uploading the correct assignment! It isn’t necessarily the most recent assignment. We carefully choose which assignments to PR, so please double-check.
What exactly should you upload? Unless we indicate otherwise, to keep the amount of work manageable, we will focus on your code. If you started in Examplar, you probably have three files (code, tests, and common). You should provide a self-contained program, i.e., everything in code as well as anything in common used by the code. The best thing to do would be to first construct that composite file, then run you tests and make sure the tests still pass: this confirms that you have a coherent, complete file. The delete those tests, and upload what is left.However, if you have a modest number of tests that exemplify each function, it can be helpful to leave those in, since they aid someone trying to read and make sense of your function! Often, it’s easier to form an idea of the function from its name and a few examples than from its body.
2.2 Reviewing
Once you upload, your work will be sent to three other students to read and review. In turn you will get three other students’ work to review.
You have the time listed on Assignments in which to write a review. We have provided you with a rubric for reviewing. For most of you this is an entirely new skill, so think of it less as “getting the right answer” and more as a learning experience.
Always be professional. Zero tolerance for anything that is not.
Don’t be perfunctory. Take some time to do a good job. Others are doing the same for you.
Don’t use GenAI to write the reviews. We can tell. (For instance, you actually did the same problem, so your responses will sound quite different from GenAI’s responses.)
Do not be verbose! Write just what needs to be said. It is rare to need more than 1–3 paragraphs, unless you have very specific things to point out (in which case feel free to write more). You will not be rewarded for verbosity; you will be penalized for it if we feel it’s not contributing value.
2.3 Reflecting
After the reviewing deadline elapses, you will find three reviews. We expect you to read them thoughtfully and distill what you learned, again using a rubric we’ve provided.
Please do not bother with false praise. A handful of reviews will be outstanding, a handful pretty bad, and most will be middling. They will contain some useful comments, mainly helping you understand how your code looks through others’ eyes. A few will contain some real nuggets of wisdom. Focus on what you learned, but you don’t have to be effusive unless someone wrote a review that really made a difference. (Even then there’s no point being effusive, becuase only we will see the reflection.)
If you get a review that you feel is not professional, you can flag it for our attention.
3 Advice from Others
The following text is freely adapted from notes by Prof. Joe Politz, who did his PhD at Brown on peer-review.
Things you liked about the code.
Any parts of code you don’t fully understand.
Suggestions for improving the code. Indicate whether these are meant to improve correctness, readability, performance, etc.
In all of this communication, remember to be polite and professional. Focus on giving crisp feedback (length alone is not a virtue: the length should be proportional to the amount of content) and writing clearly. A huge part of the job of a working software professional or researcher is accurately communicating about code and system behavior, and doing so in a way that is about the system and not about any specific person.
Some tricks for this: avoid statements that reference the author of the code, frame negative feedback as possible improvements or ways your expectations were violated, and take responsibility as the reviewer for anything you don’t understand. You can see some examples in Politz’s notes (search for “Examples”).
Finally, here’s how industrial programmers judge during code reviews.