-
Notifications
You must be signed in to change notification settings - Fork 197
Elaborate on managing Conda python, channels #606
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 6 commits
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
a6aa81b
creating new a2 myx conda
48f4766
Adding Setup_conda.md
deb69a5
rmoved .ds_store file
680d5c7
rm rhsitory
68dfbb7
split up and adding citation
f1ca239
Merge branch 'main' into CSC301_winter_myx
emma0925 b51ea48
adding reference to 5 tasks
48e70ca
Merge branch 'CSC301_winter_myx' of https://github.com/learning-softw…
e1f1e9a
rm citation
4d34763
Merge branch 'main' into CSC301_winter_myx
emma0925 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,68 @@ | ||
## Creating Pojects | ||
Conda provides a convenient way to manage dependencies and environments for your projects. A project in Conda is a directory that contains your project code along with a configuration file (environment.yml) specifying the dependencies required to run the project. | ||
|
||
### Creating the Project's file | ||
emma0925 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
Start by creating a directory for the project. It can named it whatever you like, for example: | ||
|
||
 | ||
|
||
Create an environment.yml File: Inside the project directory, create an environment.yml file. This file will specify the dependencies for the project. Here's an basic example: | ||
|
||
 | ||
|
||
#Add more dependencies as needed | ||
|
||
### Creating our enviroment | ||
|
||
Create Conda Environment: Once you've defined their dependencies, you can create a Conda environment using the conda env create command: | ||
|
||
 | ||
|
||
After creating the environment, activate it using the conda activate command: | ||
|
||
 | ||
|
||
### Creating our Python application | ||
|
||
Once the project environment is activated, the user can work on the project as usual, using the installed dependencies. The user can install additional packages, update existing ones, or remove packages as needed. Conda will manage the environment to ensure consistency and reproducibility across different systems. | ||
|
||
 | ||
|
||
Update Conda Environment: After adding the new dependencies to environment.yml, the user need to update the Conda environment. Activatethe project environment using conda activate if it's not already activated, and then run: | ||
|
||
 | ||
|
||
Once the update is complete, the user can verify that the new dependencies have been installed by checking the environment: | ||
|
||
 | ||
|
||
|
||
After updating the dependencies, it's a good practice to test the project to ensure that everything is working as expected with the new dependencies. | ||
|
||
If the user is working in a version-controlled environment, don't forget to commit the changes to environment.yml to keep track of the updates | ||
|
||
Creating projects with Conda is a straightforward process that enables the user to manage dependencies and environments effectively. By defining project dependencies in an environment.yml file, the user can ensure that your project is reproducible and easily shareable with others. Conda's environment management capabilities simplify the process of setting up and managing project environments, making it an ideal choice for Python developers. | ||
|
||
### Updating our project with new dependencies | ||
|
||
Edit environment.yml: Open the environment.yml file in project directory and add the new dependencies under the dependencies section. For example: | ||
|
||
|
||
## Viewing command-line help | ||
|
||
Conda provides comprehensive command-line help to guide users through various tasks and operations. Here's how you can access and utilize command-line help in Conda: | ||
|
||
### Viewing General Help | ||
|
||
To access general help and get an overview of available commands and options, the user can use the conda --help command. This command displays a summary of Conda's main commands and provides guidance on usage. | ||
|
||
 | ||
|
||
conda --h will do the same effect as conda --help which will return a list of conda help list. | ||
|
||
 | ||
|
||
To get help of a specific command, try running conda xxx(the command the user wish to work on) -h. For example: conda create -h, which will output the help menu of the specific command of conda create: | ||
|
||
 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
## Creating Custom Channels | ||
|
||
Before proceeding, ensure that you have conda-build installed. If not, the user can install it with the following command: | ||
|
||
 | ||
|
||
Next, organize all the packages the user wish to include in subdirectories based on the platforms intend to serve. Here's an example structure: | ||
|
||
 | ||
|
||
Run conda index command on the root directory : | ||
|
||
 | ||
|
||
The conda index command generates a repodata.json file in each repository directory, which contains metadata for the packages in the channel. | ||
|
||
To test custom channel, the user can serve it using a web server or via a file:// URL to the channel directory. The user can then test by sending a search command to the custom channel. For example, if the user want to search for files in the custom channel location /opt/channel/linux-64/, the user can use the following command: | ||
|
||
 |
85 changes: 85 additions & 0 deletions
85
Topics/Development_Process/Conda/Managing_Python using Conda.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,85 @@ | ||
## Managing Python using Conda | ||
Conda is a versatile package and environment management system designed to simplify the process of managing Python environments and dependencies. With Conda, you can create isolated environments with specific Python versions and packages, facilitating project reproducibility and compatibility. | ||
|
||
### Viewing a list of available Python Versions | ||
Conda provides users with the ability to view a list of available Python versions that can be installed within Conda environments. This feature is useful for selecting the appropriate Python version for specific project requirements or ensuring compatibility with existing codebases. Different codebases might conduct different version of python. By checking out the list of python version, conda gives you a easier and convenient way to manage your codesbases. Here's how to you view the list of available Python versions with Conda by typing `conda search python`: | ||
|
||
 | ||
|
||
Running this command will display a list of available Python versions along with their respective package versions. The output will include information such as the Python version number, package build number, and channel from which the package is available. | ||
|
||
 | ||
|
||
## Installing a different version of Python: | ||
Conda can create a new python environment without overwriting the current one, which gives the you a convenient way to use different python environment with different projects you are working on the same time in the same machine. | ||
|
||
With previous step of finding all the python version available, the first step of creating a new environment would be determine the python version you would wish to work on: | ||
|
||
 | ||
|
||
In this given example, `example` would be the name of the new environment and `python=3.10.8` would be the version of the python you would wish to work on the new environment. | ||
|
||
Activate the environment you created: | ||
|
||
 | ||
|
||
This command activates the example environment, ensuring that subsequent Python commands use the Python version installed within this environment. | ||
|
||
Verify that the new environment is your current environment. | ||
Note: In MacOS terminal, the user can check the front of the command line to check the enviroment. | ||
|
||
 | ||
|
||
To verify the current environment’s version of the python, run this in the terminal: | ||
|
||
 | ||
|
||
The output in the terminal would be the current version of the python in your current environment. | ||
|
||
## Installing Pypy | ||
|
||
To install PyPy, a fast, compliant alternative implementation of the Python language. It is a drop-in replacement for the stock Python interpreter, CPython. Whereas CPython compiles Python to intermediate bytecode that is then interpreted by a virtual machine, PyPy uses just-in-time (JIT) compilation to translate Python code into machine-native assembly language. To install you can follow these steps: | ||
|
||
Add conda-forge Channel by typing `conda config --add channels conda-force`: | ||
|
||
 | ||
|
||
This command adds the conda-forge channel to the list of channels that Conda searches for packages. | ||
|
||
Set Channel Priority by typing `conda config --set channel_priority strict`: | ||
|
||
 | ||
|
||
This command sets the channel priority to strict, ensuring that Conda prioritizes packages from the channels in the order they are listed. This helps avoid unexpected package conflicts. | ||
|
||
Create pypy Environment and activate pypy: | ||
|
||
 | ||
 | ||
|
||
It creates a new Conda environment named `pypy` and installs PyPy into it. The pypy package provided by the conda-forge channel will be installed into this environment. Then it activates the newly created pypy environment, allowing you to use PyPy for Python development within this environment. | ||
|
||
### Using a different version of python | ||
To use a different version of Python in the current environment, you'll typically create a new environment with the desired Python version and activate it. Here's how you can do it using Conda: | ||
|
||
Create a New Environment: Use the conda create command to create a new environment with the desired Python version. For example, to create an environment named myenv with Python version 3.9, you can run: | ||
|
||
 | ||
|
||
Replace myenv with preferred environment name and 3.9 with the desired Python version. | ||
|
||
Activate the New Environment and Verify the Python version.: Once the environment is created, activate it using the conda activate command. To verify that you are now using the desired Python version, you can run. | ||
|
||
 | ||
|
||
 | ||
|
||
### Updating Python: | ||
|
||
This command will updating the latest version of python. | ||
|
||
 | ||
|
||
If you want to install a minor change, not the newest one, or there is a specific version of python you want use: | ||
|
||
 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
## Managing Virtual Packages | ||
|
||
"Virtual" packages are injected into the conda solver to allow real packages to depend on features present on the system that cannot be managed directly by conda, like system driver versions or CPU features. Virtual packages are not real packages and not displayed by conda list. Instead conda runs a small bit of code to detect the presence or absence of the system feature that corresponds to the package. The currently supported list of virtual packages includes: | ||
|
||
 | ||
|
||
### Listed detected Virtual Packages | ||
To view the list of detected virtual packages using the terminal, type `conda info` in the terminal: | ||
|
||
Output would be a list of info, virtual packages would be under virtual packages section: | ||
|
||
 | ||
|
||
### Overriding detected packages | ||
To override virtual package detection for troubleshooting purposes, you can use environment variables. Supported variables include: | ||
|
||
CONDA_OVERRIDE_CUDA: Set this variable to the CUDA version number you want to override the detection with. If you want to indicate that no CUDA is detected, set it to an empty string (""). | ||
|
||
CONDA_OVERRIDE_OSX: Set this variable to the OSX version number you want to override the detection with. If you want to indicate that no OSX is detected, set it to an empty string (""). | ||
|
||
CONDA_OVERRIDE_GLIBC: Set this variable to the GLIBC version number you want to override the detection with. This only applies on Linux. If you want to indicate that no GLIBC is detected, set it to an empty string (""). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
## Viewing command-line help | ||
|
||
Conda provides comprehensive command-line help to guide users through various tasks and operations. Here's how you can access and utilize command-line help in Conda: | ||
|
||
### Viewing General Help | ||
|
||
To access general help and get an overview of available commands and options, the user can use the conda --help command. This command displays a summary of Conda's main commands and provides guidance on usage. | ||
|
||
 | ||
|
||
conda --h will do the same effect as conda --help which will return a list of conda help list. | ||
|
||
 | ||
|
||
To get help of a specific command, try running conda xxx(the command the user wish to work on) -h. For example: conda create -h, which will output the help menu of the specific command of conda create: | ||
|
||
 | ||
|
||
|
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.