Indexify CLI
Indexify CLI is a command line tool to start Executors that run functions, and to build docker images with dependencies for running them.
Start Executor
indexify-cli executor --server-addr <server-ip>:<server-port>
This will start an executor that connects to the server at the given address. The server address can be found from the Indexify server’s configuration.
Default Server Address: localhost:8900
Install the SDK
The tools to build, test and package extractors are available through the Python package indexify-extractor-sdk. A typescript SDK is under development also.
pip install indexify-extractor-sdk
Running Extractors
Running extractor locally
indexify-extractor run-local <extractor_file:ClassNameOfExtractor> --text "hello world"
This will invoke the extractor in extractor_file.py and create a Content with hello world as the data.
If you want to pass in the contents of a file into the payload use --file </path/to/file>
Running the extractor to continuously extract ingested content
You can run the extractor as a long running process to continuously receive stream of content and extract them from the Indexify control plane. You can run as many instances of the extractors you want and achieve scalability.
indexify-extractor join-server --coordinator-addr localhost:8950 --ingestion-addr localhost:8900
The addresses here can be found from Indexify server’s configuration.
From Packaged Containers
Extractors can be deployed in production with ease using docker along with Indexify server in a cluster. You can test an extractor packaged with docker by running docker locally.
docker run tensorlake/minilm-l6 run-local --text hello
Passing a local file from your laptop to docker requires bind-mounting the file into the container
docker run -v /path/to/local/file:/tmp/image.jpg tensorlake/yolo-extractor run-local yolo_extractor:YoloExtractor --file /tmp/image.jpg
The name of the extractor module is retrieved from an environment variable of the container EXTRACTOR_PATH
Was this page helpful?