Skip to content

Home

Jump to bottom
Damian Mrowca edited this page Oct 5, 2017 · 11 revisions

Welcome to the cs375 wiki!

Welcome to a brief introduction into Google Cloud Computing and how to setup your environment for tfutils. This tutorial will cover the basics needed to work on Google Cloud computing instances. This tutorial assumes you have cloned this repository and all referenced relative paths are relative to the root directory of this repository.

Before we begin

We will setup one Google Cloud project per group consisting of one instance that is preconfigured and has everything installed that is needed to complete this class. After groups have been formed, every student will receive an invite to their respective project.

We strongly encourage you to work through the excellent CS231n tutorials on how to use Google Cloud if you haven’t used it before:

Google Cloud Setup: http://cs231n.github.io/gce-tutorial/

Google Cloud GPU Setup: http://cs231n.github.io/gce-tutorial-gpus/

Please note, Google Cloud Computing is not for free! The entire class is sharing the same budget! Always stop your computing instances if you are not using them! To guarantee fairness, we will assign the same initial amount of google cloud credits to every group, and disable your cloud access if you exceed it. If you ran out of credits and need more credits please email us why.

Connecting to your Google Cloud computing instance

  1. Click on the google cloud invitation link that was send out by us and sign up or login with your Stanford account.
  2. Download the Google Cloud SDK on your computer from https://cloud.google.com/sdk/docs/ and install it by following the instructions.
  3. Click on Compute Engine in the side navigation bar, select your preconfigured instance and press Start at the top of the page to launch it. Wait until your instance is launched before trying to connect to it.
  4. Connect to your machine using
    gcloud compute ssh --zone=us-west1-b --project=<YOUR-PROJECT-ID> <YOUR-INSTANCE-NAME>.
    <YOUR-PROJECT-ID> can be found when clicking on your project name at the top navigation bar and should be something like “cs-375-group-x". <YOUR-INSTANCE-NAME> can be found after opening the Compute Engine window by selecting Compute Engine from the side navigation bar and should be something like "instance-1".
  5. Congrats you are now connected to your machine!

Configuring your environment and database setup

  1. In your home directory, clone our course repository using git clone https://github.com/neuroailab/cs375.git
  2. Download assignment1 from our course github using
    git pull
    from the course repository if you have already cloned the repository before the assignment was published.
  3. To setup your mongodb have a look at the provided example
    /mongodb/mongodb.conf
    with your favorite text editor. dbpath specifies where your database will be stored, logpath where mongodb will store it’s log file, and port the port through which you will be able to connect to the database. The port is set to 24444 which you should remember as you might need it later. Please do not modify the mongodb.conf as it is configured to launch a database that includes pretrained AlexNet model weights.
  4. To launch the mongodb database in the background execute
    sudo mongod --config=/mongodb/mongodb.conf --quiet&.
  5. Done! MongoDB should be running and be ready to accept your inputs.

Running your first model with tfutils:

  1. Open the tfutils example script
    ~/cs375/2017/assignment1/train_imagenet.py
    with your favorite text editor.
  2. Finish all coding questions.
  3. Run
    python train_imagenet.py
    within the ~/cs375/2017/assignment1 folder.
  4. Done! AlexNet should be training on ImageNet once everything has been completed successfully.

Setup to visualize your results in jupyer notebook:

  1. Execute
    jupyter notebook --generate-config.
  2. Copy the provided jupyter configuration file on the image into your home directory cp /jupyter/jupyter_notebook_config.py ~/.jupyter/jupyter_notebook_config.py.
    This configuration file configures jupyter to launch a notebook at port 10101.
  3. In your home directory execute
    mkdir notebooks
    and then
    cd notebooks.
  4. Launch your juypter notebook with
    jupyter notebook&.
  5. Copy the provided example jupyter notebook into notebooks:
    cp ~/cs375/2017/assignment1/cs375-assignment-1.ipynb ~/notebooks/cs375-assignment-1.ipynb
  6. In your internet browser go to <EXTERNAL_IP>:10101 and open the provided jupyter notebook. You can find your EXTERNAL_IP next to the instance name under Compute Engine in Google cloud computing.
  7. Done! You can now edit the example jupyter notebook and complete the assignments.
Clone this wiki locally