Script Ta Ville (the Python way)

• Ressources
  
  • Tutoriel Python http://docs.python.org/tutorial/
    
  • Allen B. Downey, Think Python, How to Think Like a Computer Scientist http://www.greenteapress.com/thinkpython/thinkpython.html
    
  • Software Carpentry http://software-carpentry.org/
    
  • Zed A. Shaw, Learn Python The Hard Way, http://learnpythonthehardway.com
    
  • Mark Pilgrim, Dive Into Python, http://diveintopython.org
    
• This should not be a course, this should be fun, hands-on and interactive
  
• At the end of the workshop, you should be able to run scripts to display open data
  
• Objectives
  
  • Automate: "Be lazy!"
    Computers are (still) stupid and ideal for repetitive tasks: they allow to process large amounts of data systematically
  • Reuse your code (and other people's)
    
  • Become autonomous in learning: solve another problem, learn another language
    
  • Learn how to control your computer, instead of being limited to programs thought and written by others
    
    • Program or be programmed, Douglas Rushkoff http://rushkoff.com/program/
      
  • Understand what can be programmed and how to do it
    The goal is not to learn Python, but a modern and powerful programming language
  • Ensure the repetability and traceability of your data processing
    
• Method and principles
  
  • Iterative development: "Have fun!"
    
    • "do the simplest thing that could possibly work", then refine ("refactor")
      
    • agile software development methods ("eXtreme Programing")
      
    • "premature optimization is the root of all evil", Don Knuth http://en.wikipedia.org/wiki/Program_optimization
      
    • test the code interactively in the interpreter, manipulate directly the variables
      
  • Do not repeat yourself: never duplicate code!
    
  • Programming style
    
    • Choose one and stick to it
      
    • Use explicit names
      
    • Avoid comments: if you need long comments, it means your code is to complex and should be rewritten
      
  • Attention to details and patience are necessary
    
    • recognize differences: a character can make a difference
      
    • be rational: computers are stupid and follows your instruction: if it is not doing what you want, YOU made a mistake
      

• Monty Python
  http://xkcd.com/353/
• Advantages
  
  • Object-oriented: re-use code
    
  • Open source: free to use and distribute
    
  • Flexibility
    
  • Largely used with lots of resources (documentation): automate VISSIM, Aimsun and ArcGIS
    
  • Easier to read and learn than other languages: read and understand what you did 6 months ago, being read by someone else
    
  • Multi-plateform, glue language: bindings for many other languages and libraries
    
  • Scientific libraries, visualization, GIS
    http://www.scipy.org, http://matplotlib.sourceforge.net, http://code.enthought.com/projects/mayavi/, http://www.sagemath.org/, http://www.gis.usu.edu/~chrisg/python/, http://www.rise-group.org/risem/clusterpy/index.html
• Weaknesses: speed (interpreted language), lack of some statistical/engineering, numerical toolboxes (as exists in Matlab)
  
• Description
  
  • Interpreted language and interpreter (shell)
    
  • Code = text file with extension .py
    Comment lines start with #
  • Scripts and modules (libraries: code reuse)
    
  • Large standard library: "batteries included"
    http://docs.python.org/library/

• Typing is important
  
• Typing is made easier by completion and command history
  
  • advanced command line: IPython, Web Notebook ($ ipython notebook --pylab inline)
    
• Evaluate expressions and variables
  
• help
  
• Some examples
  

  print "Hello World"
  s = "Hello World"
  print s
  print(s)
  name = raw_input('Hello, what\'s your name? ')
  # raw_input becomes input in Python 3
  plot(random_sample(10)) # uses matplotlib functions
  

• First script: create a file script1.py and write
  

  name = raw_input('Hello, what\'s your name? ')
  print "Hello ", name
  

  Run it with %run script1.py in IPython, or $ python script1.py on the command line

• Data types
  

  type('Hello')
  type(4)
  type(4.5)
  type(True)
  type([]) # empty list
  type([2,3]) # list
  type({}) # empty dictionary
  type({1: 'sdfasd', 'g': [1,2]}) # dictionary
  
  a = range(10)
  doubles = [2*x for x in a]
  squares = [x*x for x in a]
  

• Scripts, functions and modules
  

  #!/usr/bin/env python
  
  import matplotlib.pyplot as plt
  
  def square(x):
      return x*x
  
  values = range(100)
  squares = [square(x) for x in values]
  plt.plot(values, squares)
  

• "Workspace" and variable scope
  
• Read files
  

  f = open('2009.csv')
  f.readline()
  f.close()
  f = open('test.txt','w')
  f.write('1st line\n')
  f.write('2nd line\n')
  f.close()
  f = open('test.txt','a') # open in write mode and append at the end of the file
  

http://donnees.ville.montreal.qc.ca/fiche/velos-comptage/

• Try with the matplotlib function csv2rec (from the module matplotlib.mlab)
  

  dimanche 15 févr 2009;139;0;0;0;0;0;0;0;577
  

• One solution: use the csv module
  
• Problem with spaces and empty "cells"
  

  jeudi 02 avr 2009;;0;0;0;0;0;0;0;2 029
  

• Create datetime.date objects
  
• Plot!
  
• Reuse the code and write a function
  
• Example scripts ./read-bike-counts1.py ./read-bike-counts2.py ./read-bike-counts.py
  

This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.