MENU
| Whimsical Blog
  1. Home
  2. education
  3. Installing Stable Diffusion on Ubuntu 24.04

Installing Stable Diffusion on Ubuntu 24.04

education
mamushi

Ubuntu 24.04 has been released and Stable Diffusion Web UI is installed. Note that Ubuntu24.04 was also available in the Microsoft Store for Windows in a WSL version. The method is the same here: Ubuntu is the Desktop version and the Python version was 3.12.3 at the time of installation. Here is how to check.

python3 -V

However, Python 3.12.3 fails to install Stable Diffusion Web UI. So we decided to install pyenv, which can manage multiple Python versions on Ubuntu. pyenv allows us to easily switch between different Python versions for different projects without affecting the whole system. This makes it easier to manage situations where multiple Python versions are mixed together.

Here is how to install and manage another Python version while keeping Python 3.12.3. We will use pyenv here.

1. Installing pyenv

First, install pyenv. I run the following command, which installs the necessary dependency packages before installing pyenv on an Ubuntu system. These packages are required to compile different versions of Python via pyenv.

 sudo apt update
 sudo apt install -y build-essential libssl-dev zlib1g-dev libbz2-dev \
 libreadline-dev libsqlite3-dev wget curl llvm libncurses-dev \
 xz-utils tk-dev libffi-dev liblzma-dev git 

curl https://pyenv.run | bash

Here is a brief description of what each package is used for.

  • build-essential: contains a C/C compiler and related tools ( make, etc.); essential for building Python from source.
  • libssl-dev: required to support the SSL/TLS protocol. Used for secure network communications.
  • zlib1g-dev: compression library, used by Python modules such as gzip and zipfile.
  • libbz2-dev: bzip2 compression library, used by Python’s bz2 module.
  • libreadline-dev: A library to improve command line input. Used by the interactive Python shell.
  • libsqlite3-dev: library for SQLite database support, used in Python’s sqlite3 module.
  • wget, curl: Tools for downloading files from the Internet.
  • llvm: A low-level virtual machine (LLVM) compiler. It may be used to optimize a specific Python version or to build a specific library.
  • libncurses-dev: Library supporting console handling and interface building.
  • xz-utils: tools to support the LZMA compression algorithm, used by Python’s lzma module.
  • tk-dev: toolkit to support Tkinter, used for developing GUI applications.
  • libffi-dev: external function interface, used for calling C functions.
  • liblzma-dev: LZMA compression library, used by Python’s lzma module.
  • git: Source code version control system.

2.Editing the configuration file

  1. Open a terminal and edit the .bashrc file using the following command (if the shell you are using is bash). If you are using zsh, edit the .zshrc file.
    nano ~/.bashrc
  2. Add the following line to the end of the file
export PYENV_ROOT="$HOME/.pyenv"
[[ -d $PYENV_ROOT/bin ]] && export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init -)"

Save the changes and close the editor. To apply the changes, restart the shell or run the following command to load the configuration
source ~/.bashrc

Now pyenv is properly configured and will automatically load on startup.

What are environment variables?

Environment variables are variables managed by the operating system that contain information about the program’s execution environment. This includes system settings and data to control the program’s operation. Through environment variables, programs can receive configuration information from the operating system or other programs and operate properly based on it.

Roles of Environment Variables

  1. Provides configuration information:
    • Environment variables provide configuration information necessary for system and application operation, such as file system locations, user preferences, and system library paths.
  2. Sharing information between programs:
    • Environment variables can be used to share information between different programs. For example, an environment variable set by one application can be read and used by another application.
  3. User-specific customization:
    • Users can customize environment variables to suit their own needs and adjust program behavior.

Specific examples

  1. PATH:
    • One of the most common environment variables, it holds a list of directories where the operating system should look for programs. When executing a program from the command line, it looks for the path set in this PATH environment variable.
  2. HOME:
    • Environment variable that holds the path to the current user’s home directory. Programs use this variable to access user-specific files and settings.
  3. LANG:
    • This environment variable is used to set the language environment (locale) used by the system and programs. This allows programs to display messages in the appropriate language.

What is the main purpose of setting environment variables?

The reason for setting environment variables in the ~/.bashrc file is to ensure that certain environment settings are automatically loaded each time the shell environment is invoked. Environment variables here are system-wide configuration information used to customize system and application behavior.

  1. Set Path: Set a path to tell the system where a particular program is installed. For example, in the case of pyenv, the directory of its executable files is added to the PATH environment variable so that the pyenv command can be used from any directory.
  2. Customize program behavior: You can use environment variables to control how a program runs with different settings and options. For example, display debug messages or enable/disable certain features.
  3. Environment-specific settings: Use environment variables to apply different settings in different environments, such as development, test, and production environments.

Advantages of setting environment variables in ~/.bashrc

  • Automation: Environment variables are automatically set each time you open the terminal. This eliminates the need to manually set them each time.
  • User-specific settings: Since ~/.bashrc is a per-user file, the settings apply only to that user’s environment. You can customize your environment to meet the needs of individual users without affecting the system as a whole.

3. Installing Python

Use pyenv to install a new Python version. For example, if you want to install Python 3.10.6, run the following command

pyenv install 3.10.6

4. Setting the Global Version

Set the default Python version. Leave the existing Python 3.12.3 as is on your system and set the new version for each project. To change the version to be used globally, use

pyenv global 3.10.6

You can set it as the version to be used locally ( default in a specific directory )

pyenv local 3.10.6

5. Verifying the Version

After configuration is complete, check the Python version you are using.

python --version

The display driver is configured because we turned on the option to install third-party drivers when installing Ubuntu 24.04. The additional driver “nvidia-driver-535” was selected. On this computer, the nvidia-smi command showed 12.2. The number “12.2” displayed by nvidia-smi probably refers to the NVIDIA driver version.

nvidia-smi (NVIDIA System Management Interface ) is a command line tool for monitoring and managing the status and configuration of NVIDIA GPUs. With this tool, you can get detailed information about installed NVIDIA GPUs and view real-time performance data such as current GPU utilization, temperature, and memory usage.

Examples of information provided by nvidia-smi:

  • GPU model number anddriver version: Shows the model number of the installed GPU and the NVIDIA driver version it is using. (This is 12.2)
  • GPU utilization: Shows the GPU’s compute core and memory utilization in percentages.
  • Temperature: Displays the GPU temperature in degrees Celsius.
  • Memory Information: Displays total memory, memory in use, and free memory.
  • Process Information: Displays the ID and memory usage of each process using the GPU.

This tool is especially useful in datacenter and server environments, as well as in machine learning and deep learning research that uses GPUs, to monitor GPU health and manage resources. nvidia-smi is also suitable for use in automated environments, as it outputs information in a form that can be easily integrated with batch scripts and system monitoring tools.

Next, I would like to install the CUDA Toolkit, but the official NVIDIA page did not yet indicate Ubuntu 24.04. (Because Ubuntu 24.04 has not been released for a long time.) So I enter the following command to see how to install it.

nvcc -V

since the CUDA Toolkit is not installed,

command, ‘nvcc’, is not found. You can install it with the following method: sudo apt install nvididid
sudo apt install nvidia-cuda-toolkit

You will probably be able to install it with this. This will probably install it. To check which version is installed, type the wrong command on purpose. For example, nvcc- v (this command does not exist). You will then see the ‘nvcc- v’ command, the ‘nvcc- v’

Command, ‘nvcc-‘ not found. Could it be:
What is …?

and you can check the version.

So, install CUDA Toolkit with the following command.

sudo apt install nvidia-cuda-toolkit

After installation, check the version.

nvcc -V

12.0 seems to have been installed.

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Fri_Jan__6_16:45:21_PST_2023
Cuda compilation tools, release 12.0, V12.0.140
Build cuda_12.0.r12.0/compiler.32267302_0

Now that you are ready to install Stable Diffusion, create a GitHub clone of the Stable Diffusion web UI on your own computer.

https://github.com/AUTOMATIC1111/stable-diffusion-webui

git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git

Create a virtual environment with the previous version of python.

  1. Create a virtual environment: To create a virtual environment, use venv (a standard Python module).
    python -m venv myenv
    where myenv is the name of the virtual environment and can be changed arbitrarily.
  2. Activating the virtual environment: To activate the created virtual environment, execute the following command
    source myenv/bin/activate
    This will activate the virtual environment, allowing you to safely install and use Python and other packages within the environment.
  3. Terminating use of the virtual environment: To terminate use of the virtual environment, execute the following command
    deactivate

You are now ready to use the virtual environment with Python 3.10.6.

Why a virtual environment?

  1. Avoid dependency conflicts:.
    • Different projects often require different library versions. Installing all libraries directly on one system can cause conflicts between versions. By using a virtual environment, different projects can isolate different libraries and their versions without affecting each other.
  2. Environment reproducibility:.
    • Virtual environments allow you to keep track of exactly which libraries and their versions you are using and easily reproduce the same settings for other developers and environments. This is very useful during team development and project transitions.
  3. System Protection:.
    • Installing directly on the entire system risks accidentally modifying or destabilizing critical parts of the system. By working within a virtual environment, you can protect key systems and only modify or recreate the virtual environment if something goes wrong.
  4. Easily remove and rebuild environments:.
    • When a project is finished or no longer needed, the virtual environment can be easily deleted. This prevents unwanted programs from remaining on the system and keeps the system “clean”.

Thus, virtual environments are an important tool for improving development efficiency, making your system easier to organize, and making it safer to work on a variety of projects. It is especially recommended for those who do not want to mess up their computers.

The actual work done is as follows

git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git
cd stable-diffusion-webui
python -m venv myenv
source myenv/bin/activate

Then install Stable Diffusion Web UI.

./webui.sh

You may see the following during execution

Cannot locate TCMalloc. Do you have tcmalloc or google-perftool installed on your system?

This warning message indicates that the application could not detect TCMalloc (Thread-Caching Malloc), which is part of Google Performance Tools and improves memory usage efficiency over the standard memory allocator. than the standard memory allocator, thereby improving application performance. However, this message is a warning, and the application itself is working fine using the standard memory allocator.

If improved performance is required, or if you wish to eliminate this warning, you can consider installing TCMalloc. Below are the instructions for installing TCMalloc on Ubuntu.

How to install TCMalloc (Ubuntu)

  1. Install package: sudo apt update
    sudo apt update
    sudo apt install libgoogle-perftools-dev
    This command will install the libgoogle-perftools-dev package, which also includes TCMalloc.
  2. Application Configuration:.
    • You will need to adjust the application configuration or set environment variables as needed so that the application uses TCMalloc. Please refer to the application documentation for this part.

After installing TCMalloc, restart the application and see if the warning message goes away. You should see an improvement in memory usage and possibly a performance increase.

Note that if you do not need the performance improvement, or if the application is running normally, you can ignore the warning.

In fact, I later realized that running the script (webui.sh) in the stable-diffusion-webui directory included the ability to automatically create and activate a Python virtual environment.

The script details the creation and activation of the virtual environment in the following part

if [[ $use_venv -eq 1 ]] && [[ -z "${VIRTUAL_ENV}" ]]; 
then 
 printf "\n%s\n" "${delimiter}" 
 printf "Create and activate python venv" 
 printf "\n%s \n" "${delimiter}" 
 cd "${install_dir}"/"${clone_dir}"/ || { printf "\e[1m\e[31mERROR: Can't cd to %s/%s/, aborting... \e[0m" "${install_dir}" "${clone_dir}"; exit 1; } 
 if [[ ! -d "${venv_dir}" ]] 
 then 
 "${python_cmd}" -m venv "${venv_dir}" 
 first_launch=1 
 fi 
 # shellcheck source=/dev/null 
 if [[ -f "${venv_dir}"/bin/activate ]] 
 then 
 source "${venv_dir}"/bin/activate 
 else 
 printf "\n%s\n" "${delimiter}" 
 printf "\e[1m\e[31mERROR: Cannot activate python venv, aborting... \e[0m" 
 printf "\n%s\n" "${delimiter}" 
 exit 1 
 fi 
else 
 printf "\n%s\n" "${delimiter}" 
 printf "python venv already activated or run without venv: ${ VIRTUAL_ENV}" 
 printf "\n%s\n" "${delimiter}" 
fi

This code segment does the following

  1. If the virtual environment has not yet been created or there is no active virtual environment already ( "${VIRTUAL_ENV}" is empty), the virtual environment is created.
  2. After the virtual environment has been successfully created, activate it.
  3. If activation of the virtual environment fails for any reason, the script outputs an error message and exits.

The script automatically handles many functions at runtime, including reading necessary environment variables and settings, installing the appropriate libraries for the environment, and performing additional configuration based on GPU information. It also checks the GPU information using the nvidia-smi andlspci commands, and based on that information, installs the specific PyTorch version, and performs other optimal settings based on the hard-case configuration.

Note 1.
This section describes how to deal with the situation where you have installed Python 3.10.6 using pyenv and then want to use the system’s default Python version (e.g., 3.12.3). pyenv allows you to specify a specific Python version without overriding the system’s default Python version. You can use pyenv to specify a specific Python version without overriding the system’s default Python version.

How to revert to the default Python version

  1. Clear global settings:.
    • To clear the global Python version set by pyenv (the default version at the user level), run the following command
      pyenv global system
    • This command reverts the pyenv controlled version specification to “system”, i.e., the default Python version preinstalled on your system.
  2. Confirmation: pyenv global system
    • To check the currently active Python version, use the following command
      python --version (python3-V)
    • You can then check if pyenv is using “system” (the default Python version).

Specifying a Version for a Specific Project

If you want to use a different Python version for each project, you can use the pyenv local command within that project’s directory to set a specific version. For example, if you want to use Python 3.10.6 in one project, run the following command in the root directory of the project

pyenv local 3.10.6

This will cause Python 3.10.6 to be used only when working in that directory, and the system default Python version will be used everywhere else.

You can use pyenv to list the installed Python versions. To do this, use the following command

pyenv versions

This command will list all Python versions installed on the system, with the currently active version marked with an asterisk (*). For example, you might get output like this

system
* 3.7.7 (set by /home/user/.pyenv/version)
3.8.2
3.9.0

Here, “system” refers to the system’s default Python, and the other entries are versions installed through pyenv. The active version (here Python 3.7.7) is marked with an asterisk next to it.

Note 2.
Multiple application versions can be managed using Git, which is particularly well suited for software versioning and is widely used to effectively manage multiple versions and features. The following are the main methods of version control using Git

Using Branches

The branching feature of Git is very effective for developing different versions of a project at the same time. Branches allow you to develop new features or fix bugs independently of the main line of development (usually the master or main branch). Each branch is independent of the others, allowing different versions of the application to be maintained simultaneously in the same repository.

Using Tags

You can use Git’s tagging feature to mark a particular commit as a version. This is useful for marking releases and important checkpoints. For example, you can create a tag for a release of version 1.0, 2.0, etc. to easily reproduce that status.

About Disk Space

Creating multiple virtual environments consumes a lot of disk space because dependent libraries and project files are replicated in each environment. However, with Git, branches and tags are very lightweight and actually use very little additional disk space because all data is stored in a single database. Git stores only the diffs of changes, so even managing multiple versions consumes minimal disk space.

Conclusion

With Git, you can efficiently manage different feature developments and multiple application versions, while saving disk space. This eliminates the need to have duplicate resources for each environment, as in a virtual environment. It is possible to balance the need for version control with resource optimization.

Note 3.
There are several factors to consider when choosing between using Python via WSL or creating a Python virtual environment directly in the Windows command prompt. There are advantages and disadvantages to both methods, and the final choice depends on the needs of the project, the environment setup, and personal preference.

Advantages and disadvantages of using WSL

Advantages

  • Linux environment: Allows native use of Linux-based tools and scripts. This is especially useful when using tools that only work or are optimized for Linux.
  • Development Consistency: If you are using a Linux server as your production environment, your development environment can be consistent with the production environment.

Disadvantages

  • Performance: WSL uses virtualization technology, so certain operations, such as file system operations, may degrade performance.
  • Setup complexity: WSL configuration and administration may be more complex than the Windows command prompt.

Advantages and Disadvantages of Using Python Virtual Environment with Windows Command Prompt

Advantages

  • Easy setup: Python and the virtual environment can be easily configured directly on Windows, no special configuration is required.
  • Performance: No overhead due to virtualization since Python runs directly on Windows without WSL.

Disadvantages

  • Tool compatibility: Some development tools and libraries may not work well on Windows or are only available on Linux.
  • Environmental differences: If your production environment is Linux, developing on Windows may cause problems due to environmental differences.

Conclusion

  • Project Dependencies and Requirements: If your project has Linux-specific dependencies or is designed to run in a Linux environment, WSL is a good choice.
  • Performance and simplicity: If performance is important and Linux-specific tools are not required, then building a virtual environment in a Windows command prompt is the way to go.

Ultimately, we recommend making your choice based on which environment your project will run most efficiently and is easiest to manage.

education
Please share if you like it!
TOC