Installation

You can install the Python package with

$ pip install interpretableai

The interpretableai package is a wrapper around the Julia implementation of IAI algorithms, so you need to configure a working Julia installation in addition to installing the Python package from PyPI. There are two options for doing this.

Option 1: Automatically install Julia and IAI

The interpretableai package can automatically download and configure a working Julia installation with the IAI system image. To do this, simply run the following commands in Python:

import interpretableai
interpretableai.install_julia()
interpretableai.install_system_image()

After running these commands, you will need to restart Python.

Option 2: Using a separate Julia installation

Follow the instructions for installing the IAI system image to get a working Julia installation with the IAI modules included.

In order to use Julia to run the IAI algorithms, Python also needs to know where to find Julia on your computer. By default, it will look for an executable on the system PATH named julia. It is also possible to specify the location of Julia manually if preferred.

Once installed, you need to run the following command in Python to configure the connection to Julia for access the IAI algorithms. This only needs to be run the first time following installation of the interpretableai package:

import interpretableai
interpretableai.install()

Using the Python package

After configuring Julia following one of the two options above, you can begin using the interpretableai package. You can access the IAI algorithms in any Python session by loading the package with the following command:

from interpretableai import iai

The first time the package is loaded, you will receive a message that there is no license file, along with a machine ID. Please follow the instructions to obtain a license file.

After this, all of the package functionality is available under the iai module.

Troubleshooting

Below is a list of errors you might receive when setting up the package and how to resolve them. If none of the suggestions fix the problem, you can also refer to the PyJulia troubleshooting guide.

Julia not correctly configured or not on the PATH

When running interpretableai.install(), you may receive the following error if Julia is not correctly configured:

julia.core.JuliaNotFound: Julia executable `julia` cannot be found.

If you have installed Julia, make sure Julia executable is in the
system PATH.  Alternatively, specify file path to the Julia executable
using `julia` keyword argument.

If you have not installed Julia, download Julia from
https://julialang.org/downloads/ and install it.

When running from interpretableai import iai, you may receive the following error if Julia is not correctly configured:

FileNotFoundError: [Errno 2] No such file or directory: 'julia': 'julia'

On Windows only, you may also see a message similar to the following about being unable to load a library:

WARNING: Error during initialization of module PCRE:
ErrorException("could not load library "libpcre2-8"
The specified module could not be found.
")

In each case, you will either need to add julia to the system PATH, or follow the steps to specify the path to Julia.

IAI not present in Julia installation

You may see the following error when importing the package:

IAI is not present in your Julia installation.

Make sure you have followed the instructions for installing the IAI system image so that the IAI module is present in your Julia installation.

Architecture mismatch between Python and Julia

On macOS with ARM (M-series) processors, you may see an error similar to the following when importing the package:

OSError: dlopen(/Applications/Julia-1.8.app/Contents/Resources/julia/lib/libjulia.1.8.dylib, 0x000A):
tried: '/Applications/Julia-1.8.app/Contents/Resources/julia/lib/libjulia.1.8.dylib'
(mach-o file, but is an incompatible architecture (have (arm64), need (x86_64)))

This indicates that the architecture of the Julia executable (indicated by have in the error message) does not match the architecture of the Python executable (indicated by need). You will need to reinstall Julia and/or Python to ensure they have the same architecture, either Intel (x86_64) or ARM (arm64).

Crashes due to mismatched versions of binary dependencies

Julia and Python both depend on a number of binary dependencies (shared libraries/dylibs/DLLs), and some of these dependencies are used by both languages. In most cases, only a single version of a library can be loaded at one time in a single process, and so the version of a library that is loaded first by either language will be the version used by both languages. This poses a problem when the two languages use different versions of the library that are incompatible with each other (e.g. an older library from Python may be missing functions expected from a newer version of the library used by Julia, or vice versa). If this happens, it usually results in a segmentation fault (or similar abrupt crash).

Here are some real examples of this occuring and the resulting errors:

• An error while loading a library that a specific procedure/function could not be found:

    fatal: error thrown and no exception handler available.
    InitError(mod=:LibCURL_jll, error=ErrorException("could not load library "libcurl-4.dll"
    The specified procedure could not be found. "))
    ...

  This was caused by a mismatch in the versions of libssh2 that prevented libcurl from loading (even though the versions of libcurl matched exactly).

• An indefinite hang during initialization:

    Updating registry at `~/.julia/registries/General`
    ┌ Error: curl_multi_socket_action: 8
    └ @ Downloads.Curl /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36

  This was caused by a mismatch in the versions of libcurl.

• A segmentation fault or access violation during loading:

    Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
    Exception: EXCEPTION_ACCESS_VIOLATION at 0x7ffaf941b941 -- RtlMoveMemory at C:\Windows\SYSTEM32\ntdll.dll (unknown line)in expression starting at none:0
    RtlMoveMemory at C:\Windows\SYSTEM32\ntdll.dll (unknown line)
    RtlFreeHeap at C:\Windows\SYSTEM32\ntdll.dll (unknown line)
    free_base at C:\Windows\System32\ucrtbase.dll (unknown line)
    RC4_options at C:\Users\iai\miniconda3\envs\iai\Library\bin\libcrypto-3-x64.dll (unknown line)
    RC4_options at C:\Users\iai\miniconda3\envs\iai\Library\bin\libcrypto-3-x64.dll (unknown line)
    SSL_free at C:\Users\iai\.julia\artifacts\9822b36d99275836cdcd2079edbdb458deb07a0c\bin\libssl-3-x64.dll (unknown line)free at C:\Users\iai\builds\InterpretableAI\SystemImage\SysImgBuilder\.julia\packages\OpenSSL\8wxMC\src\ssl.jl:306 [inlined]
    ...

  This was caused by a mismatch in the versions of libopenssl.

The most likely libraries to cause this problem when using the Python interface are libcurl,libssh2, and libopenssl. If there is no indication as to what might be causing the error, you can try enabling Python's faulthandler to see more information about a segmentation fault (either by setting the PYTHONFAULTHANDLER environment variable before starting Python, or by calling faulthandler.enable() inside Python).

If you run into this problem, the first thing you should try doing is disabling the automatic update check in IAI, which you can do by setting the environment variable IAI_DISABLE_UPDATE_CHECK=true before loading IAI:

import os
os.environ['IAI_DISABLE_UPDATE_CHECK'] = 'true'

If the issue persists, then you should try to align the versions of the mismatched dependencies as best you can. You can discover the versions of dependencies used by Julia in the following ways:

• for dependencies built-in to Julia (e.g. libcurl and libssh2), you can refer to the Julia source code for the version of Julia you are using. For instance, we can find that Julia 1.10.4 uses libssh2 v1.11.0 from this link.

• for dependencies not built-in to Julia (e.g. libopenssl), you can query the version using the Julia package manager. For instance, we can check the version of libopenssl by running the following in Julia package mode:

    pkg> st -m OpenSSL_jll
    Status `~/.julia/environments/v1.10/Manifest.toml`
    ⌃ [458c3c95] OpenSSL_jll v3.0.13+1

Once you know the version of the dependency that Julia is using, you should try to alter the version that Python is using to be as close as possible. For conda-based Python installations, you can modify the versions using conda install, e.g. to set libssh2 to v1.11.0 you can run

conda install libssh2=1.11.0

You may need to try multiple close versions of libraries to find one that works.

If you are unable to get this working, a last-resort resolution can be to pre-emptively load the Julia versions of libraries before Python can load its versions, following the procedure described here.

Advanced: Specifying location of Julia

If julia is not on the system PATH, or you would like to use another Julia executable, there are two additional ways to specify the location of Julia.

If you are unsure of the path to your Julia installation, you can get the path to the julia executable by running the following command in Julia:

julia> unsafe_string(Base.JLOptions().julia_bin)

1. Specifying location of Julia using IAI_JULIA

You can set the environment variable IAI_JULIA to point to the julia executable, e.g. in Python:

import os
os.environ['IAI_JULIA'] = 'path/to/julia'

You can then proceed with installing and importing the package as usual, making sure that this variable is defined in every Python session before importing iai.

2. Specifying location of Julia manually

If you would like to specify the location of Julia without using environment variables, you can pass the path to a Julia executable to install:

import interpretableai
interpretableai.install(runtime='path/to/julia')

You will also need to take an additional step in order to manually initialize the connection to Julia with the path to julia as the runtime argument. This will need to be done in every Python session before importing iai:

from julia import Julia
Julia(runtime='path/to/julia')  # and any other parameters as needed
from interpretableai import iai

Advanced: Specifying location of system image

If you do not replace the default Julia system image, there are two additional ways to specify the location of the system image.

1. Specifying location of system image using IAI_SYSTEM_IMAGE

You can set the environment variable IAI_SYSTEM_IMAGE to point to the location of the system image, e.g. in Python:

import os
os.environ['IAI_SYSTEM_IMAGE'] = 'path/to/sys'

You can then proceed with installing and importing the package as usual, making sure that this variable is defined in every Python session before importing iai.

2. Specifying location of system image manually

If you would like to specify the location of the system image without using environment variables, you will need to manually initialize the connection to Julia with the path to the system image as the sysimage argument. This will need to be done in every Python session before importing the iai module:

from julia import Julia
Julia(sysimage='path/to/sys')  # and any other parameters as needed
from interpretableai import iai

Advanced: Development builds of interpretableai

Development builds of the interpretableai package are uploaded to the Test PyPI repository, and can be installed with the following command:

$ pip install --upgrade --extra-index-url https://test.pypi.org/simple/ interpretableai

There is no need to use these development builds unless directed to do so.