Demonstration RIVA Docker Container

Modified on Fri, 14 Nov at 2:47 PM

TABLE OF CONTENTS

(Acoustic Speech Recognition & Text To Speech)
Steps
Conclusion

(Acoustic Speech Recognition & Text To Speech)

Riva is an excellent tool that provides GPU-accelerated acoustic speech recognition and text-to-speech services. It also supports translation and other conversational pipelines. When paired with USB audio hardware (microphone and speaker), Riva can be integrated with a demonstration LLM to enable fully offline audio input/output interfaces. More details about Riva can be found at: https://www.nvidia.com/en-us/ai-data-science/products/riva/

This recipe is a work in progress. The following steps outline the current process for setting up the Riva Docker container on Cardshark.

Steps

Step 1: Set the Power Mode and Reboot (if needed)

Set the device power mode to ensure optimal performance:

sudo nvpmodel -m 0

Step 2: Setup NGC CLI

Download and configure the Nvidia NGC CLI tool. You will need an API key from Nvidia (available at https://org.ngc.nvidia.com/setup/api-key).

wget --content-disposition https://api.ngc.nvidia.com/v2/resources/nvidia/ngc-apps/ngc_cli/versions/3.58.0/files/ngccli_arm64.zip -O ngccli_arm64.zip && unzip ngccli_arm64.zip
find ngc-cli/ -type f -exec md5sum {} + | LC_ALL=C sort | md5sum -c ngc-cli.md5
chmod u+x ngc-cli/ngc
echo "export PATH=\"\$PATH:$(pwd)/ngc-cli\"" >> ~/.bash_profile && source ~/.bash_profile
ngc config set

Step 3: Install and Configure Docker

Follow these steps to install Docker and configure it for use with Nvidia’s container runtime.

a. Install Docker prerequisites and add the official Docker GPG key:

sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

b. Add the Docker repository and install Docker packages:

echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

c. Verify the Docker installation:

sudo docker run hello-world

d. Configure Docker to run without sudo and enable the services:

sudo usermod -aG docker $USER
sudo systemctl enable docker.service
sudo systemctl enable containerd.service

e. Configure the Docker daemon to use the Nvidia runtime:

sudo apt install nano
sudo nano /etc/docker/daemon.json

Insert the following JSON configuration into the file:

{
    "default-runtime": "nvidia",
    "runtimes": {
        "nvidia": {
            "path": "/usr/bin/nvidia-container-runtime",
            "runtimeArgs": []
        }
    }
}

Step 4: Install Riva Dependencies and Reboot

Install the required Nvidia Jetpack packages:

sudo apt install nvidia-jetpack nvidia-jetpack-runtime nvidia-jetpack-dev

For the changes to take effect, reboot the device:

sudo reboot

After rebooting, verify the Docker runtime configuration:

docker info | grep -i runtime

Step 5: Setup Riva

Create a working directory for Riva and download the Riva Quickstart image. (Note: At the time of this writing, version 2.17.0 is recommended.)

cd ~/
mkdir riva-setup
cd riva-setup
ngc registry resource download-version nvidia/riva/riva_quickstart_arm64:2.17.0
cd riva_quickstart_arm64_v2.17.0/
nano config.sh

Edit the config.sh file to enable the desired services. For example, you might set:

service_enabled_asr=true
service_enabled_nlp=false
service_enabled_tts=true
service_enabled_nmt=false

Before proceeding, ensure that your USB audio hardware is connected.

Make the initialization scripts executable and run the setup:

chmod +x riva_init.sh
chmod +x riva_start.sh
chmod +x riva_stop.sh

./riva_init.sh
# Carefully review the output before continuing
./riva_start.sh

Conclusion

Further refinements and documentation will be added as the setup process evolves.