Assignment: Networking

This assignment describes how to use basic networking with a focus on concepts most useful to robotics. First we will learn about pipes in Bash or a unix shell. You can do these exercises by sshing into your drone. When you type at a shell, each shell program reads from a stream, known as “standard input” and writes to two output streams, “standard output” and “standard error.” When you “print” in python, it writes its output to standard output. In another language, such as C, you use other functions, such as “printf” to write to standard output. We will ask you to write short programs in python as part of the assignment. You can do the assignment on your drone’s Rasberry Pi once you can SSH into the drone. To do this you will need to at least be able to power your PI and connect to it via the base station.

Problem 1: Standard Output (10 points)

  1. Write a python program that prints "Hello world" to standard output. Use sys.stdout to print.

    Answer: helloworld.py
    #!/usr/bin/env python
    import sys
    
    sys.stdout.write("Hello world\n")
    
    

  2. Use the program echo to print things to standard output. Submit the command line that you used.

    Answer: echo Hello World

Problem 2: Standard Input (10 points)

In addition to writing to standard output, a program can read from standard input. The program cat, short for concatentate, reads from standard input and writes the result to standard output.

  1. Run cat with no arguments. First think about what might happen. Then type text at the command line and observe the result. What is happening? Why?

    Answer:
    When you type (once you type Enter), cat reads the typed text from standard in. You also see an echo of what you typed on the console. Then it writes what you typed to standard out. So you'll see what you typed appear twice on the console.
  2. Modify your program to read characters from standard input, and print them to standard output when you press "Enter."

    Answer: stdio.py
    #!/usr/bin/env python
    
    import sys
    
    while True:
        text = sys.stdin.readline()
        sys.stdout.write(text)
    
    

Problem 3: Pipes (20 points)

Now we will discuss pipes. You can use pipes to redirect standard input, standard output, and standard error. First, you can use “>” to direct standard output to a file. For example, “echo Hello World > test.txt” will write the text “Hello World” to “test.txt”. The program “cat test.txt” can be used to view the file.

  1. Create files "one.txt", "two.txt" and "three.txt" that contain the strings "1", "2", and "3" respectively using echo and output redirect. Verify your work with ls and cat.

    Answer: files.sh
    #!/bin/sh
    
    echo 1 > one.txt
    echo 2 > two.txt
    echo 3 > three.txt
    ls
    cat one.txt
    cat two.txt
    cat three.txt
    
    
  2. By convention, almost all shell programs read input from standard input, and write their output to standard output. Any error messages are printed to standard error. You can chain shell programs together by using "|". For example, the program ls writes the contents of a directory to standard output. The program sort reads from standard input, sorts what it reads, and writes the sorted content to standard output. So you can use ls | sort to print out a sorted directory list. Read the man page for sort (man sort) to learn how to sort in reverse order. Modify the pipe script above to print the contents of a directory in reverse alphabetical order.

    Answer:
    ls | sort -r

  3. Use cat, | and echo to print "hello world." Do not write to any files and use both commands one time.

    Answer:
    echo Hello World | cat
  4. This is not the simplest way to print hello world. Can you suggest a simpler way? (We asked you to do it the more complicated way to practice with pipes.)

    Answer:
    echo Hello World
  5. Now modify your python script from Problem 2 to read from standard input, sort each line alphabetically in reverse order, and print the output. That is, your script should be have like sort -r. Note that you will have to read the entire contents of standard input into memory in order to sort it.

    Answer: reversesort.py
    #!/usr/bin/env python
    
    import sys
    
    lines = sys.stdin.readlines()
    
    for l in reversed(sorted(lines)):
        sys.stdout.write(l)
    
    

Problem 4: Standard Error (10 points)

In addition to standard input and standard output, there is a third stream, standard error. If there is an error in a chain of pipes, it will be printed to the terminal rather than buried in the input to the next program.

  1. Try running ls -a | sort to see all files in a directory sorted. Redirect the output to a file named sorted.

    Answer: ls -a | sort > sorted.txt

  2. Now add the nonexistant "e" option to the ls command: ls -a | sort > sorted.txt. What happens to the file sorted.txt?

    Answer:
    $ ls -ea | sort > sorted.txt
    ls: invalid option -- 'e'
    Try 'ls --help' for more information.
    

  3. Modify your sort program to print status reports to standard error, but print its output to standard out. Redirect the output to a file and look at the file. Use it to sort the output of ls -a instead of using sort as above. If your program is correct, you should see your status messages printed when you run it, and see only the sorted output written to the sorted.txt file.

    Answer: reversesortstatus.py
    #!/usr/bin/env python
    
    import sys
    sys.stderr.write("Reading stdin.\n")
    lines = sys.stdin.readlines()
    sys.stderr.write("Sorting.\n")
    sorted = reversed(sorted(lines))
    sys.stderr.write("Writing.\n")
    for l in sorted:
        sys.stdout.write(l)
    
    

Problem 5: Networking (20 points)

Now we will explore the command nc to learn more about basic networking. The command nc is short for “netcat” and is similar to cat but works over network connections. It reads from standard input and writes its contents not to standard output, but to a specified server.

  1. Point nc to google.com: nc www.google.com 80 When you first connect, it will be silent. Then type any arbitrary text and press enter. What happens?

    Answer: You should get HTTP/1.0 400 Bad Request along with an error message in the body. However you did successfully connect to the server and send bits to it! So you know that the basic network connection is up.

  2. Now type some valid http into nc: GET / HTTP/1.1

    Answer: You should get the html context of google.com.

  3. Now use nc to make a server. In one window, type nc -l 12345. This will cause nc to listen on port 12345. In another terminal on the same machine, type nc localhost 12345. You can type a message in one window and it will appear in the other window (and vice versa). This trick can be very useful to test basic internet connectivity - if the client and server can send packets at all.
  4. How can we know what servers are out there? Well, some are "common knowledge", like google.com on port 80. By convention, roscore listens on port 11311. Try using nc to connect to port 11311 on a machine where roscore is running, such as the PI. What happens? What protocol is roscore using to communicate?

    Answer: You should get an error message about a bad HTTP request. The program roscore uses XMLRPC carried over HTTP.

  5. Another useful tool is nmap, which scans through a range of ports (and optionally, through a range of IP addresses) and reports information.
    1. Run nmap localhost on your Pi. What ports are open? Look up each port and submit what it does.
    2. Run nmap with and without the nc -l 1234 command running from above. What is the difference? Why?

    3. Answer: When the nc -l 1234 is running, nmap will detect a server running on port 1234. However it nc is not running, nmap will not detect a server.
    4. Run nmap with roscore running. Does nmap report roscore? Why or why not? Use man nmap to find command line options for nmap that report the ROS port 11311.

      Answer: nmap localhost -p1-65535
    5. Portscan google.com. What ports are open? Why?

      Answer: nmap google.com for me returns port 80 and 443. Port 80 is used for serving http, and 443 is for serving https (encrypted/private http connections). Google only opens those ports because they want to serve web pages (so they can give you search results, to make money). They don't open other ports because they don't want hackers breaking into their system, and every service you run on a port is a potential hack attempt.

Problem 6: Talking to Your Robot (10 points)

Now we are going to talk about the public internet. So far we’ve been doing our work using mostly localhost, the local machine we are connected to, and google.com. The most common situation is that your base station machine (your laptop or desktop) and your robot are connected over TCP/IP to the same local network. Then you can look up your machine’s IP address (ifconfig in Unix; other ways in other OSes), and your robot’s IP address, and connect them. How can you find your robot’s IP address? Well it’s a chicken-and-egg problem. If you knew the IP address, you can connect to the robot and run ifconfig and find the IP address, but you don’t know the IP address!

What to do? There are several solutions.

  1. Brainstorm how you can solve the chicken-and-egg program to connect to your robot.

    Answer: We can't answer this question without knowing the network setup. I am right now connected over a cable with internet connection sharing in Ubuntu. Then I used nmap to scan the local IP range in the subnet that my laptop created. If you have access to a router, you can use the router to look up who is connected, and find the robot's IP address that way. You can also plug in a USB keyboard and HDMI monitor into the PI in order to see what its IP address is configured as, and to set up its network settings. The usual practice is to save these settings so that when it boots, it does the right thing, so that you can connect using only the base station in the future.

  2. So how does the internet work? A computer typically uses the DHCP protocol to request an IP address from a server that manages the local network. At your house, it's likely to be your cable modem or home router. At Brown, CIS manages the routers that keep the network up. Once you have an IP address, you are on the internet. When I was a student at MIT, the internet connection in my dorm room gave me an 18.*.*.* address that was on the public internet with no firewall. I could run any server that I wanted, and many students did (and do!). However there are serious security concerns with this setup. People could serve bad stuff, like SPAM, or they could get hacked by bad actors who would *use* the connection to serve SPAM. So it's safer to not give people public IP addresses and most people don't. There aren't a lot of them either - one of the things you pay your home ISP for is a public IP address, and you usually only get one. To try out DHCP, connect to Brown, Brown_Guest, and RLAB. REport back your IP address each time using your operating system. Then connect again. Do you get the same address or a different address?

    Answer: The specific IP addresses will vary based on the network. Most DHCP servers will give the same machine the same IP address every time it connects, but there is no guarantee this will be the case.

  3. How can we have more than one device connected to the Internet? The usual answer is a protocol called Network Address Translation. This remaps the IP address space so that you can have one public IP address that usually connects to a router. Then the router has a public (WAN or wide-area-network) side with the public IP address) and a private (LAN or local-area network) with multiple connections. The IP addresses on the private side are not full-fledged IP addresses because they cannot act as servers. You can't listen on a port from the private side and connect to it from the public internet. However you can do private-to-private connections, and many people do, e.g., for games or robots! You can also selectively open a connection to the public internet on many routers using port forwarding. This can be configured on the router; most routers offer a web-based API to configure these kinds of remappings. You can say port 11311 on the public side maps to a particular IP address and port on the private side, for example. Under a typical NAT setting, the robot and the base station will typically both connect to the router via DHCP to obtain an IP address. Their IP address will be in the 192.168.*.* range, or the 10.*.*.* range, both by convention used for private local networks. The router's public IP address will be whatever it is, and both machines will have internet access through NAT. However neither machine will be a server to the public internet. But that's okay - they only need to be servers to each other. So they can listen on ports and server request using their local (192.168 or 10.0.0.*) IP addresses. Connect to the Brown_Guest, RLAB, and Brown networks. For each network, answer the following questions:
    1. What IP address do you have on each network?
    2. What is the router's IP?
    3. What ports are open on the router?
    4. Use nmap to identify the machines on each network. How many are there?

Problem 7: Look Ma, No Internet! (10 points)

But what about if there is no public internet connection? What if you want to fly your drone in the wilderness? Well, there does exist cellular modems and sattellite connections, but you can also tell your drone to act as a Wifi Hotspot. It can create a network and run a DHCP server. You can configure this on your drone using the file /etc/hostapd/hostapd.conf.

Then you can connect your laptop’s base station using the SSID and passphrase specified in that file, and connect to the drone.

Alternatively you can set up your laptop as the Wifi base station and configure the drone to connect to its network. The details will vary depending on your laptop OS and settings.

Your Pi is configured to be a Wireless AP Master by default. Connect to it with your base station.

  1. Which machine is acting as the DHCP server?
  2. What is the Pi's IP address? What is yours?
  3. What is the ping time between you and the Pi when you are close to the Pi
  4. How far away can you get from the Pi before it starts disconnecting?
  5. What is the ping time when you are far away from the Pi?

Problem 8: Design Scenario 1 (5 points)

You want to fly your drone autonomously for 10 miles while viewing the camera output continuously from your base station. Design a solution that will allow you to do this. What hardware, software, and network access providers do you need? Report the weight, CPU computation, and cost this capability will add to your drone.

Answer:

  • Buy a PiAnywhere card.
  • Buy a prepaid SIM card to obtain an IP address
  • Configure the image server and rosbridge interface to work over the cellular uplink.
  • Connect to the Javascript interface via the cellular link.

Problem 9: Design Scenario 2 (5 points)

You want to connect to the drone over Wifi but also be able to connect to the internet in the CIT using the RLAB network. Design a solution that will allow you to do this. What hardware, software, and access providers do you need? Report the weight, CPU computation, and cost this capability will add to your drone.

Answer:

There are two alternatives. Both require purchasing something like a USB wifi card like this one.

Option 1: Connect the Wifi to the Rasberry Pi. Put one Wifi card on RLAB and the second one acting as a Wireless Master, as it is now. Set up “bridging” on the drone so that the drone itself is a router, and forwards internet packets back and forth to the base station, over two separate Wifi hops.

Option 2: Connect the Wifi to the base station. Configure the drone to act in Wireless Managed mode and to connect to the base station. Configure the base station as the wireless master, and set it up to run a DHCP server. (This is often done by setting up “Internet Connection Sharing” from your OS gui.)

Duckietown solved this problem at MIT using Option 1. They bought a USB Wifi card and put it on each robot. Then they configured the Rasberry Pi to act as a router.

Handin

When you are done, use this link to create your Networking Github Repo. Commit and push the relevant files (networking.pdf, and any scripts you wrote throughout the assignment) to this Github Repo before the deadline.