Installing binaries

Getting Started
1

Hello, this is my first post here!

I am having difficulties to install a binary for IPOPT, that I need for using with PYOMO.
Could you provide me some suggestions.
Some specific suggestion regarding my special goal.
Or some general tip on how to do this kind of thing?
Below is a bit more explanation on what I tried to do.

Thanks

Michel

I have been doing things like this (or similar using !wget):

if os.path.isfile("/data/notebook_files/ipopt"):
    print("Ipopt already installed")
else:
    url = "https://github.com/maajdl/Coin-solvers/raw/main/coin.linux-intel64.20230221.tgz"
    urllib.request.urlretrieve(url, "coin.tgz")
    file = tarfile.open('/data/notebook_files/coin.tgz')
    file.extractall()
    file.close()
    os.chmod("/data/notebook_files/ipopt", stat.S_IEXEC)
    print("Ipopt is now installed")

I got it working at some point in time, using the url shown above.
But now it doesn’t work anymore!
I can’t see why!
The error message provided by PYOMO is this:

WARNING: Failed to create solver with name ‘ipopt’: Failed to set executable
for solver ipopt. File with name=/data/notebook_files/ipopt either does
not exist or it is not executable.

2

Hi and welcome!

Please check the output for !ls -la ./ - is ipopt actually present? What permissions are set?

Best regards,
Igor

3

Thanks a lot igro!

Ideed, this was about permissions!
Believe it or not, but I had explained my problem to ChatGPT,
and that was also the suggestion it gave me! :grinning:

Before that, I had been trying so many different things and so quickly that I was lost.
Now it works and it is reproducible, and I become rather confident.
A one-liner was the key:

os.chmod(f, stat.S_IEXEC | stat.S_IREAD | stat.S_IWRITE)

I still have some questions on installing binaries.
As you have seen, I have installed it in

“/data/notebook_files”

But is that the right or the best way to proceed?
Can I assume that the IPOPT binarie will remain there forever?
Do I still need to care about how to install this binary, or can I forget this story?
Do I need to include the installation code into my workbook?
Especially if I would share it at some time?
Are there some examples showing how such things should be done?
Are there some guidelines?

Another question, maybe for another thread is about installing packages (pyomo).
Should I re-install pyomo everytime I start a session?
Or is there a way to ‘install’ it permanently?

Thanks again!
And sorry that I did not give a feedback earlier.

Michel

4

Believe it or not, but I had explained my problem to ChatGPT,
and that was also the suggestion it gave me! :grinning:

Haha, cool!

os.chmod(f, stat.S_IEXEC | stat.S_IREAD | stat.S_IWRITE)

You can also write it in the following way (set +x on top of the current permissions):

os.chmod(ipopt_path, os.stat(ipopt_path).st_mode | stat.S_IEXEC)

…but it would be easier to use Python magics, Terminal or init.sh script for all the required pre-configuration – please see below for more details.

But is that the right or the best way to proceed?
Can I assume that the IPOPT binarie will remain there forever?

Data in the notebook_files (and workspace_files) folders is persisted between runs – these folders are located on a separate WebDAV storage and are mounted to a computational agent each time you start a notebook (“Attached data” → “Notebook files” in the side panel). Other files on the computation agent will be lost once the notebook is stopped (VM is disposed).

Do I still need to care about how to install this binary, or can I forget this story?
Do I need to include the installation code into my workbook?

While data in these folders is persisted, you will still need to set correct permissions each time the folder is mounted to the agent FS, but instead of this installation script you can always use console (“Tools” → “Terminal”) or Python magics (e.g. !sudo chmod +x ./ipopt) to make any necessary changes.

Another option I’d recommend going with – is to create init.sh file in the notebook_files folder and place the following commands there to automate the process (this shell script is executed each time the computation agent is started):

sudo chmod +x /data/notebook_files/ipopt
sudo cp /data/notebook_files/ipopt /usr/local/bin/ipopt

Second command will add ipopt binary to the PATH to make it available by its name from the terminal or Python magics (e.g. !ipopt instead of !./ipopt or !/data/notebook_files/ipopt).

You can also move the binary and the init.sh script to the Workspace Files folder (and update path to /data/workspace_files/*) – this way it will be accessible from every notebook within a workspace (in “Home” workspace you will need to explicitly mount Workspace Files folder in each notebook, in other workspaces – workspace_files folder is always attached to all notebooks).

Especially if I would share it at some time?

Attached files including the init.sh script are available to all invited users.

Are there some examples showing how such things should be done?
Are there some guidelines?

Please refer to the documentation for more details:

There is also the following explanation when you create a new init.sh file using the corresponding link from the Environment tab:

# This shell script runs when the machine starts, 
# before the user-installed packages are set up.
#
# Use it to install additional dependencies.
#
# Examples:
#
# sudo apt update && sudo apt install -y some-package
#
# pip install some-package --additional-arguments
#
# The logs are stored in /tmp/log;
# the error log is copied to Notebook files.

Another question, maybe for another thread is about installing packages (pyomo).
Should I re-install pyomo everytime I start a session?
Or is there a way to ‘install’ it permanently?

I think my comments above cover those questions as well, but feel free to ask additional questions if you have any.

Thank you,
Igor

5

Thanks a lot igro
Sorry for answering so late.

I begin to feel confortable with installing binaries.

Michel