Skip to content

Developing Indexify

Install Dependencies

Before you start, this doc may be outdated. Please follow the procedur in, this is the source of truth.

Rust Compiler

Install various rust related tools -

  • Rust Compiler -
  • Cargo tools - clippy, rustfmt is very helpful for formating and fixing warnings.

Python Dependencies

Create a virtual env

python3.11 -m venv ve
source ve/bin/activate

Install the extractors

pip install --upgrade --force-reinstall .

The following workaround is needed until PyO3 can detect virtualenvs in OSX and some Ubuntu versions

 export PYTHONPATH=$PYTHONPATH:$(pwd)/ve/lib/python3.11/site-packages


Install coreutils

brew install coreutils

Running Tests

We currently depend on the Qdrant VectorDB and Postgres to test Indexify.

Start Development Dependencies

make local-dev

Run Tests

Run the unit and integration tests

cargo test -- --test-threads 1

Running the service locally

Build the Binary

Build the server in development mode

cargo build

Create a development database

make local-dev

Start the server

Once the binary is built start it with a default config -

./target/debug/indexify server  -d --config-path local_server_config.yaml

Start an Extractor

Start an extractor to join the server Clone the repository

git clone
cd indexify-extractors
indexify extractor start --coordinator-addr localhost:8950 --ingestion-addr localhost:8900 --extractor-path embedding-extractors/minilm-l6/

Visual Studio DevContainer

Visual Studio Code Devcontainers have been setup as well. Opening the codebase in VS Studio Code should prompt opening the project in a container. Once the container is up, test that the application can be compiled and run -

  1. make local-dev
  2. Install the Python Dependencies as described above.
  3. Compile and Run the application as described above.


If you're within the dev container, you can call the docker-compose-v1 from within /usr/bin/

If docker produces a EONET error, please try to build your devcontainer prior to launching it in vscode: devcontainer up --workspace-folder

To build a local container for testing, run the following command from the root of the project:

make build-container-dev