Introduction to Python - Gurobi

Page created by Don Marsh
 
CONTINUE READING
Introduction to Python - Gurobi
Introduction to Python
Introduction to Python - Gurobi
Quick reminder about questions

                Please ask them!

                     Copyright 2017, Gurobi Optimization, Inc.
Introduction to Python - Gurobi
A few words about Gurobi
Introduction to Python - Gurobi
Gurobi algorithms
 • Continuous optimization
    • LP: primal and dual simplex, sifting, parallel barrier, concurrent
    • QP: primal and dual simplex, parallel barrier
    • QCP: parallel barrier

 • Discrete optimization (MIP, MIQP, MIQCP)
    • Parallel branch-and-bound
    • Relaxations solved with continuous algorithms
    • Cutting planes and heuristics

 • General
    •   Automatic presolve
    •   Identifying irreducible inconsistent subsystems (IIS)
    •   Automated parameter tuning tool
    •   …

                                           Copyright 2017, Gurobi Optimization, Inc.
Introduction to Python - Gurobi
Interfacing with Gurobi

 • Command-line tool

 • Full-featured interactive shell

 • Matrix-oriented APIs
     • C, MATLAB, R

 • Object-oriented APIs
     • C++, Java, .NET, Python

 • Modeling systems
     • Commercial: AMPL, Frontline Solvers, …
     • Free: JuliaOpt, PuLP, Pyomo, SolverStudio, …

                                       Copyright 2017, Gurobi Optimization, Inc.
Getting started with Python
Python in a nutshell

 • Python is a powerful programming language

    •   Very easy to learn
    •   Runs on every major operating system
    •   Suitable for beginners and professional developers
    •   Clean, simple and compact syntax
    •   Open Source community: Tons of ready-made libraries and frameworks

 • Python is also a great choice for mathematical modeling

    • Many popular packages for scientific computing
    • Jupyter Notebooks: Interactive graphical environment
    • Language syntax looks similar to mathematical notation

                                       Copyright 2017, Gurobi Optimization, Inc.   7
Why Python?

• Gurobi provides a variety of programming interfaces
    •   Python
    •   Java
    •   C++
    •   .NET (C#, Visual Basic, …)
    •   MATLAB,
    •   R
    •   ...

• Gurobi is committed to all of these interfaces

• This presentation focuses on Python since it is easy to learn and use

                                     Copyright 2017, Gurobi Optimization, Inc.   8
Python installation

 • The Gurobi Optimizer installation includes a small Python 2.7
   distribution with a basic set of packages.

 • Gurobi Optimizer works with multiple Python distributions
   (including http://www.python.org).

 • In this presentation, we use Anaconda Python

     •   Download from http://www.gurobi.com/downloads/get-anaconda
     •   Windows, Mac, Linux; 32/64-bit
     •   All basic packages in the default installation
     •   Easy-to-use package management (conda)
     •   Gurobi provides/maintains Anaconda packages
     •   Python 2.7 (recommended) and 3.5 (latest)

                                        Copyright 2017, Gurobi Optimization, Inc.   9
Basic concepts

 • Python is an “interpreted language”
     • No need for a manual compiling step before running a program.
     • Performance optimization is done in the background (byte-code compilation/execution).

 • Two basic ways to use Python
     • Put your code in a text file (e.g. example.py) and run the command “python example.py”.
     • Start the Python Interactive Shell (command “python”) and enter commands line by line.

 • In this presentation we will also use Jupyter Notebook
     • A rich graphical interactive environment for Python

                                           Copyright 2017, Gurobi Optimization, Inc.             10
Jupyter Notebook

 • The Jupyter Notebook is a web application that
   allows you to create and share documents that
   contain live code, equations, visualizations and
   explanatory text.
 • Free software with an active open source
   community
 • Born out of the IPython Project in 2014
 • Already included in Anaconda Python

 • Simply change to your working directory and
   run jupyter notebook on the command-line

 • Learn more at http://jupyter.org/

                                       Copyright 2017, Gurobi Optimization, Inc.   11
Integrated development environments (IDEs)

 • Many Python IDEs available: https://wiki.python.org/moin/IntegratedDevelopmentEnvironments
    • Easy to install Gurobi in other Python distributions

 • Spyder is distributed with Anaconda
    •   Scientific Python Development EnviRonment
    •   Familiar for users of MATLAB, Visual Studio, etc.
    •   Built-in editor, file explorer, documentation browser
    •   Graphical interface for debugging (ipdb)
    •   IPython console for interactive code execution

                                           Copyright 2017, Gurobi Optimization, Inc.
Connecting Gurobi and Python

               Copyright 2017, Gurobi Optimization, Inc.
Connecting Python and Gurobi

 • Basic Python interpreter: gurobi.bat / gurobi.sh
     • Included and configured when you install Gurobi Optimizer

 • Anaconda Python – automated installation
     conda config --add channels              http://conda.anaconda.org/gurobi
     conda install gurobi

 • Manual installation for other Python distributions
     • Change to GUROBI_HOME directory
     • Run python setup.py install

 • For details, see the Gurobi Quick Start Guide: http://www.gurobi.com/documentation/

                                        Copyright 2017, Gurobi Optimization, Inc.   14
The Gurobi Python API

 • Gurobi Interactive Shell is a Python shell with Gurobi extensions
     • Gurobi distribution includes Python interpreter and basic modules
     • Nothing extra to buy or install

 • Don't need to be a Python expert to start modeling in Python

 • Powerful environment for interacting with existing models
     • Model file formats: .mps, .rew, .lp, .rlp, .ilp
     • All file formats support built-in compression: .zip, .gz, .bz2, .7z

                                         Copyright 2017, Gurobi Optimization, Inc.
Demonstration: Solve model in Jupyter Notebook

 • Goals:
    • Make yourself familiar with Jupyter Notebook
    • Solve a model file in MPS format

 • Example file: afiro.mps (included in the examples
   directory of the Gurobi installation)

 • Steps:
    • Import the gurobipy module
    • Create model object from model file
    • Solve model to optimality
    • Print solution values

                                       Copyright 2017, Gurobi Optimization, Inc.   16
Log file (default: gurobi.log)

 Gurobi 7.0.2 (mac64, Python) logging started Thu Feb           2 11:57:06 2017

 Read MPS format model from file afiro.mps
 Reading time = 0.00 seconds
 AFIRO: 27 rows, 32 columns, 83 nonzeros
 Optimize a model with 27 rows, 32 columns and 83 nonzeros
 Coefficient statistics:
   Matrix range     [1e-01, 2e+00]
   Objective range [3e-01, 1e+01]
   Bounds range     [0e+00, 0e+00]
   RHS range        [4e+01, 5e+02]
 Presolve removed 18 rows and 20 columns
 Presolve time: 0.01s
 Presolved: 9 rows, 12 columns, 32 nonzeros

 Iteration    Objective        Primal Inf.       Dual Inf.              Time
          0   -4.8565680e+02    1.363638e+02      0.000000e+00              0s
          3   -4.6475314e+02    0.000000e+00      0.000000e+00              0s

 Solved in 3 iterations and 0.01 seconds
 Optimal objective -4.647531429e+02

                                         Copyright 2017, Gurobi Optimization, Inc.
Log file

  Variable            X
 -------------------------
           X01           80
           X02        25.5
           X03        54.5
           X04        84.8
           X06     18.2143
           X14     18.2143
           X16     19.3071
           X22          500
           X23      475.92
           X24       24.08
           X26          215
           X36     339.943
           X37     383.943

                              Copyright 2017, Gurobi Optimization, Inc.
Interactive shell environment

 • Interactive shell provides access to all Gurobi Optimizer functionality through Python API

 • read() is a global function which returns an object of type Model

 • Model.optimize() and Model.printAttr() are methods (functions on the Model object)

 • We can choose any name for the Model object variable:

    model = read('afiro.mps')                            m = read('afiro.mps')
    model.optimize()                        or           m.optimize()
    model.printAttr('X')                                 m.printAttr('X')

                                      Copyright 2017, Gurobi Optimization, Inc.
Basic Gurobi concepts
 • Parameters control the operation of the Gurobi solvers. They
   must be modified before the optimization begins.
                                                                                                  Model           Attributes           Parameters
      http://www.gurobi.com/documentation/current/parameters.html

 • Attributes are the primary mechanism for querying and                                          Constraints                       Attributes

   modifying properties of a Gurobi model.
                                                                                                  Variables                         Attributes
      http://www.gurobi.com/documentation/current/refman/attributes.html

                                                                                                  Special Ordered Sets              Attributes
   Attributes can be associated with
     •   the model as a whole (e.g. the objective function value)
     •   variables (e.g. lower bounds)
     •   constraints (e.g. the right-hand side)
     •   SOSs (e.g. IIS membership)
                                                                                                  Environment            Global Parameter Settings

 • Environments are the containers for models and global
                                                                                                   Model 1                Local Parameter Settings
   parameter settings. When a model is created inside an
   environment, it creates a local copy of all global parameter
                                                                                                   Model 2                Local Parameter Settings
   settings.
 • The Python API automatically provides a default environment.

                                                      Copyright 2017, Gurobi Optimization, Inc.                                                  20
Attributes

 • Multiple ways to access attributes
     • Methods: getAttr() and setAttr()on Model, Constr, or Var objects
         • Model.getAttr()also can retrieve attributes for multiple variables or constraints
     • Directly: m.ModelName

 • Examples:
     • print(m.getAttr('ObjVal'))
     • print(m.ObjVal)

     • m.ModelName = 'My Model'
     • m.setAttr(m.ModelName, 'My Model')

                                            Copyright 2017, Gurobi Optimization, Inc.
Python API fundamentals #1
 • Important global functions
     • setParam()         set (global) parameters for new models in this session
     • read()             create model object from a model file

 • Primary objects
     • Model              the model
     • Var                a variable
     • Constr             a constraint

 • Parameter/attribute examples:
     •   setParam('TimeLimit', 60)                          Set global parameter
     •   model.TimeLimit = 60                               Set local parameter
     •   variable.VarName = 'TotalCosts'                    Set variable attribute
     •   print(constraint.RHS)                              Set constraint attribute

 • Other important methods:
     • Model.setObjective(expression)                        Set objective function
     • Model.optimize()                                      Start optimization
     • Model.reset()                                         Reset model state

                                         Copyright 2017, Gurobi Optimization, Inc.     22
Python API fundamentals #2
 • Model summary information methods
     • Model.printAttr()             display attribute values in bulk, without needing a loop
     • Model.printStats()            display sizes, coefficient statistics, etc.
     • Model.printQuality()          display info about solution quality

 • Model introspection methods
     •   Model.getVars()             get variables as a list of Var objects
     •   Model.getConstrs()          get constraints as a list of Constr objects
     •   Model.getRow()              get LHS expression for a given constraint
     •   Model.getCol()              get list of constraints for a given variable

 • Python control statements
     • for i in :
         
     • if :
         
     • Must indent body of each statement (at least one space or tab)

                                          Copyright 2017, Gurobi Optimization, Inc.             23
Getting help
 • Use the help() function with no arguments to get general help

 • You can get help on specific Gurobi objects
     •   help(Model)      Gurobi model object (or help(m) if m is a model)
     •   help(Var)        Gurobi variable object (or help(v) if v is a variable)
     •   help(Constr)     Gurobi constraint object (or help(c) if c is a constraint)
     •   help(GRB.attr)   Gurobi attributes

 • You can get help on specific functions or methods
     • Ex: help(read)
     • Ex: help(Model.printAttr)

 • You can get help on any parameter
     • Ex: paramHelp('Method')

                                        Copyright 2017, Gurobi Optimization, Inc.
Demonstration – model introspection
 • Launch interactive shell
 • Read coins model from examples/data subdirectory of Gurobi installation:
     model = read('/examples/data/coins.lp')
 • Print model statistics and solve it:
     model.printStats()
     model.optimize()
 • Display solution:
     for v in :
       if v. != 0:
         print(v., v.)

 • Notes/Hints:
     • Don't forget to indent for control structures and enter blank line to end the for loop
     • Use help(Model) to replace  with correct method
     • Use help(Var) to replace  and  with correct attributes

                                          Copyright 2017, Gurobi Optimization, Inc.
Console
 gurobi> model = read('coins.lp')
 gurobi> model.printStats()

 Statistics for model Unnamed :
   Linear constraint matrix     :   4   Constrs, 9 Vars, 16 NZs
   Variable types               :   4   Continuous, 5 Integer
   Matrix coefficient range     :   [   0.06, 7.2 ]
   Objective coefficient range :    [   0.01, 1 ]
   Variable bound range         :   [   50, 1000 ]
   RHS coefficient range        :   [   0, 0 ]

                                    Copyright 2017, Gurobi Optimization, Inc.
Console
 gurobi> model.optimize()
 ...
 Found heuristic solution: objective -0
 Presolve removed 1 rows and 5 columns
 Presolve time: 0.01s
 Presolved: 3 rows, 4 columns, 9 nonzeros
 Variable types: 0 continuous, 4 integer (0 binary)

 Root relaxation: objective 1.134615e+02, 4 iterations, 0.00 seconds

        Nodes    |      Current Node   |     Objective Bounds                         |     Work
     Expl Unexpl |    Obj Depth IntInf | Incumbent    BestBd                      Gap | It/Node Time

         0     0     113.46154   0   1      -0.00000          113.46154               -    -    0s
 H       0     0                         113.4500000          113.46154           0.01%    -    0s

 Explored 0 nodes (4 simplex iterations) in 0.03 seconds
 Thread count was 4 (of 4 available processors)

 Optimal solution found (tolerance 1.00e-04)
 Best objective 1.134500000000e+02, best bound 1.134500000000e+02, gap 0.0%

                                      Copyright 2017, Gurobi Optimization, Inc.
Console
 gurobi> for v in model.getVars():
 .......   if v.x != 0:
 .......     print v.varname, v.x
 .......
 Dimes 2.0
 Quarters 53.0
 Dollars 100.0
 Cu 999.8
 Ni 46.9
 Zi 50.0
 Mn 30.0

                                 Copyright 2017, Gurobi Optimization, Inc.
Deciphering errors

 • Errors are raised when something goes wrong
    • Trace specifies where error occurred
    • Last line specifies error type and may provide additional details

 • Ex: IndentationError raised when block spacing is incorrect (must indent before if)

        gurobi> for v in model.getVars():
        ....... if v.x != 0:
          File "", line 2
            if v.x != 0:
             ^
        IndentationError: expected an indented block

 •  corresponds to interactive shell

                                         Copyright 2017, Gurobi Optimization, Inc.
Deciphering errors

 • Ex: AttributeError raised when attribute is incorrect (should be VarName)

   gurobi> for v in model.getVars():
   .......    if v.X != 0:
   .......      print v.Name, v.X
   .......
   Traceback (most recent call last):
     File "", line 3, in 
     File "var.pxi", line 76, in gurobipy.Var.__getattr__ (../../src/python/gurobipy.c:11798)
     File "var.pxi", line 121, in gurobipy.Var.getAttr (../../src/python/gurobipy.c:12310)
   AttributeError: 'gurobipy.Var' object has no attribute 'Name'

 • Don't worry about trace from within gurobipy

                                     Copyright 2017, Gurobi Optimization, Inc.
Additional model methods
 • Attribute and parameter management
    •   Model.getAttr()           get attribute values
    •   Model.setAttr()           set attribute values
    •   Model.setParam()          set parameter values for a model
    •   Model.getParamInfo()      get current parameter values

 • Importing and exporting
    • Model.read()        import data from a file (ex: .prm)
    • Model.write() export model data to a file (ex: .sol)
    • Data type is determined by file extension

 • Copying and exporting
    • Model.copy()       create model copy, ex: n = m.copy()
    • Model.relax()      create model copy with integer variables relaxed
    • Model.fixed()      create model copy with integer variables fixed to values in the current solution

                                       Copyright 2017, Gurobi Optimization, Inc.
Additional model methods

 • Algorithms
    •   Model.optimize()     solve a model
    •   Model.reset()        discard any solution information (next solve starts from scratch)
    •   Model.presolve()     create new model containing presolved version of original model
    •   Model.computeIIS()   compute IIS for an infeasible model
    •   Model.feasRelax()    perform feasibility relaxation on a model by introducing penalty variables
                             and penalty objective

                                 Copyright 2017, Gurobi Optimization, Inc.
Editing and running Python programs
 • Python is not just an interactive shell

 • Python programs are typically stored in text files with .py extension
     • Any text editor can be used to edit them (ex: notepad)
     • Most programming editors have Python mode with syntax highlighting, etc.
     • Comments are preceded with '#' character

 • Most functionality in Python is provided by modules
     • To use a module in a program, include an import statement
     • Ex: import math

 • To use the Gurobi interface in a program, import the gurobipy module:
     from gurobipy import *

 • Use command line to run program with Python interpreter distributed with Gurobi

                                             Copyright 2017, Gurobi Optimization, Inc.
program.py
 # import gurobipy module
 from gurobipy import *

 # create empty model
 m = Model()

 # change model name attribute
 m.ModelName = 'My First Model'

 # process the pending change
 m.update()

 # export model to file
 m.write('emptymodel.mps')
 m.write('emptymodel.lp')

 # print to console
 print('This program creates an empty model named:', m.Modelname)
 print('Model exported to emptymodel.mps and emptymodel.lp')

                                  Copyright 2017, Gurobi Optimization, Inc.
Run a Python program: Console output
 $ gurobi.sh program.py
 This program creates an empty model named: My Model
 Model exported to emptymodel.mps and emptymodel.lp

                                 Copyright 2017, Gurobi Optimization, Inc.
Useful packages

 • Common questions:
    •   Can Gurobi read data from … ?
    •   Can Gurobi output the solution as …?
    •   Can Gurobi be used through a web service?
    •   Can Gurobi solve this model using an approach that … ?
    •   Can Gurobi… ?

 • The answer:
    • Yes, most of the time since Gurobi stands on the shoulders of giants
          • Python includes a large standard library
          • Many more add-on libraries and frameworks are available for download
    • The Python API is a full interface that includes all Gurobi features
    • You can use Python to build powerful, complete Gurobi applications

                                            Copyright 2017, Gurobi Optimization, Inc.
Pandas

• Fast and efficient DataFrame and Series objects
• Read/write CSV, HDF5, Excel, SQL, plain text
• Missing data handling
• Slicing, fancy indexing, subsetting
• Merges, joins
• Split-apply-combine operations (groupby)

              http://pandas.pydata.org

                                        Copyright 2017, Gurobi Optimization, Inc.
Bokeh

• Interactive visualization for web browsers
    • High performance with streaming data and big data
    • In-browser interactivity on otherwise static pages
    • Bindings for Python, Scala, Julia, R

          http://bokeh.pydata.org

                                       Copyright 2017, Gurobi Optimization, Inc.
Use our Gurobi online resources
 • The Gurobi Reference Manual is freely available at

       http://www.gurobi.com/documentation/current/refman/index.html

   It contains detailed descriptions of our algorithms and APIs.

 • We provide API examples for all major programming languages as well as
   interactive examples at

       http://www.gurobi.com/resources/examples/example-models-overview

                                               Copyright 2017, Gurobi Optimization, Inc.   39
Gurobi documentation
 • Quick Start Guides
     • One for each supported OS

 • Example Tour
     • 22 examples
     • Examples for all APIs

 • Reference Manual
     • Detailed information on APIs, parameters, attributes, …

 • Available online at http://www.gurobi.com/documentation

 • Docs are installed in the docs subdirectory
 • Sample code is installed in the examples subdirectory

                                           Copyright 2017, Gurobi Optimization, Inc.
You can also read