mirror of
https://git.isriupjv.fr/ISRI/ai-server
synced 2025-04-24 01:58:12 +02:00
109 lines
2.7 KiB
Markdown
109 lines
2.7 KiB
Markdown
# AI-Server
|
|
|
|
A server that can serve AI models with an API and an authentication system
|
|
|
|
## Usage
|
|
|
|
The ai-server project require a conda environement. You can use Docker to deploy it easily.
|
|
|
|
### Docker
|
|
|
|
You can easily use docker-compose to run the project.
|
|
Simply go into the project directory and run :
|
|
|
|
```bash
|
|
docker compose run -d
|
|
```
|
|
|
|
#### Environment Variables
|
|
|
|
The project use special environement variables :
|
|
|
|
| Name | Description |
|
|
|-----------------|-------------------------------------------|
|
|
| MODEL_DIRECTORY | the directory where the models are stored |
|
|
|
|
#### Volumes
|
|
|
|
The project might store data inside multiples volumes :
|
|
|
|
| Path | Type | Required | Description |
|
|
|------|------|----------|-------------|
|
|
| /models/ | Volume | True | The place where the models are stored |
|
|
| /root/.huggingface/hub | Bind | False | The place where the internal models are cached. Avoid redownloading huge amount of data at every inference |
|
|
|
|
## Models
|
|
|
|
A model is an object that can be loaded and do inference.
|
|
|
|
It is stored inside a directory and must always contain a `config.json` file.
|
|
|
|
### Configuration
|
|
|
|
This is a json-structured file with basic information about the model.
|
|
|
|
It describe :
|
|
- its type (see below)
|
|
- its tags
|
|
- its interface
|
|
- its inputs
|
|
- its output mimetype
|
|
|
|
And other properties depending on the model type.
|
|
|
|
#### Types
|
|
|
|
There is for now only a single type of Model : the Python model
|
|
|
|
##### Python Model
|
|
|
|
A python model is isolated in a `conda` environement.
|
|
|
|
To be considered a Python model, you need theses three files :
|
|
|
|
| File | Description |
|
|
|-------------|-------------------------------|
|
|
| config.json | The configuration file |
|
|
| env | The conda virtual environment |
|
|
| model.py | The model file |
|
|
|
|
###### Configuration
|
|
|
|
Additionnal fields might be found in the configuration :
|
|
|
|
...
|
|
|
|
###### Virtual Environment
|
|
|
|
You can create a conda virtual environement it with :
|
|
|
|
`conda create --prefix ./env/ python=<version>`
|
|
|
|
You can install your requirements inside with `pip3` or `conda`.
|
|
|
|
###### Internal Model
|
|
|
|
You need to create a `model.py` file containing a class named `Model`.
|
|
This class must be exposed with the Pyro4 library for inter-operability with our main environement.
|
|
|
|
Here is an example of an internal model :
|
|
|
|
```python
|
|
import Pyro4
|
|
|
|
import torch
|
|
import transformers
|
|
|
|
|
|
@Pyro4.expose
|
|
class Model:
|
|
def __init__(self):
|
|
# load the model
|
|
self.pipe = transformers.pipeline(...)
|
|
|
|
def infer(self, messages: list[dict]) -> bytes:
|
|
with torch.no_grad():
|
|
outputs = self.pipe(messages)
|
|
|
|
return outputs[0]["generated_text"][-1]
|
|
```
|