ShareAdd.jl
This Julia package helps to reduce clutter in your main shared environment (and thus avoid package incompatibility problems) by making it easy to use multiple shared or temporary environments. It exports two macros: @usingany
and @usingtmp
, envisioned for two different workflows. The package also provides several utility functions for managing shared environments.
Glossary
The definitions below deviate somewhat from the strict definitions of Environment, Project, and Package as given in the Julia docs, and refer to the most common and relevant cases.
- Project File: The
Project.toml
file in environment's folder. Itsdeps
section lists packages available in this env. May contain additional data like compatibility requirements. 🔗 - Manifest:
TOML
-format file in environment's folder. It contains the actual state of the environment, including all indirect dependencies and their versions. Environment can contain multiple Manifest files for different Julia versions. 🔗 - Environment: Simply a folder containing a
Project.toml
and optionally a Manifest file. Projects or packages are also environments. All packages in theProject.toml
of the active environment are available for import. 🔗 - Shared Environment: Any environment in the
path-to-Julia/.Julia/environments
folder. These environments can be addressed by prepending@
to the env name, without typing it's whole path. Typically shared environments do not contain projects. 🔗 - Main / Default Environment: The subfolder in the environments folder, named according to the Julia minor version , e.g. for Julia v1.11 it's name will be
v1.11
. This is the default env upon start of Julia, except Julia was started with some specific project. Especially for novices it is common to work in the default environment and install all packages therein. This may result in compat conflicts. The motivation to createShareAdd
was to help avoiding this kind of situaltion. LOAD_PATH
: An array of currently available environments. The default value is["@", "@v#.#", "@stdlib"]
, where"@"
refers to the current project, and"@v#.#"
stands for the "main" env. Environments can be added both in the form of env folder paths, or, for shared envs, as the env name prepended by@
- e.g.["@", "@v#.#", "@stdlib", "@MyTools"]
.- Stacked Environments: A concept of using multiple environments at the same time, so that the packages from each environment are available. Realized by having multiple (shared) envs in the
LOAD_PATH
. This is the main mechanism behindShareAdd
. 🔗. - Project: A directory containing a
Project.toml
file as well as source and other relevant files. Project is a subtype of environment. - Package: Is a project adhering to a certain structure, which contains source files and other resources, suitable for distribution. Packages can be loaded with
using
orimport
. 🔗
@usingany
macro
This macro makes package(s) available, if they are not already, and loads them with using
keyword.
- If a package is available in an environment in
LOAD_PATH
, that's OK. - If a package is available in a shared environment, this environment will be pushed into
LOAD_PATH
. - Otherwise if it can be installed, you will be prompted to select an environment to install the package(s).
- If the package is not listed in any registry, an error will be thrown.
# simplest usage case
@usingany SomePackage
For more usage options see @usingany
docs .
@usingany
usage example
Let's assume, while working on your package MyPackage
, we temporarily need packages TOML
, Plots
, and Chairmarks
. However, they shouldn't be added permanently to your package dependencies. Furthermore, from the package BenchmarkTools
we need only the macro @btime
and the function save
. We also need Unitful
, which is already an installed dependence of MyPackage
.
TOML
is available in the stdlib
, Plots
and BenchmarkTools
you already put into a shared environment @utilities
, and Chairmarks
is not on your computer yet.
Now, first, you add ShareAdd to your "main" (standard) enviroment, making it available at all times:
]
(YourEnv) pkg> activate
Activating project at `~/.julia/environments/v1.11`
(@v1.11) pkg> add ShareAdd
(...)
(@v1.11 pkg> activate . # back to your environment
(YourEnv) pkg>
By that occasion you may also want to clean your standard environment: It is generally not recommended having a lot of packages there.
Now, the only thing you need is to type into REPL (or adding to your script) the following lines:
using ShareAdd
@usingany Unitful, TOML, Plots, Chairmarks
@usingany BenchmarkTools: @btime, save
As Chairmarks
was not installed yet, you will be asked as to where to install it. You may e.g. add it to your existing @utilities
shared environment, or let create a new environment @Chairmarks
and put it there.
Afterwards @utilities
(and @Chairmarks
, if created) will be added to LOAD_PATH
, making their packages available.
Finally, the macros will execute using Unitful, TOML, Plots, Chairmarks
resp. using BenchmarkTools: @btime, save
- and that's it. Enjoy!
@usingany
with updates
By setting the corresponding kwarg, it is possible to first update the packages and/or environments prior to execution of import. E.g. the following command would update the packages Pkg1
, Pkg2
in their shared environments:
using ShareAdd
@usingany update_pkg = true Pkg1, Pkg2
@usingany
without explicitly calling @usingany
ShareAdd.jl
can be combined nicely with BasicAutoloads.jl
. See this Discourse post to learn how get packages silently loaded if you call their functions in the REPL - e.g. if you type mean([1,2,3])
or 1.55u"V"
.
Versioned manifests
If your currently used Julia version supports versioned manifests (i.e. >= v1.10.8), then on any updates using ShareAdd
package (see ShareAdd.update
), a versioned manifest will be created in each updated env. The function ShareAdd.make_current_mnf
can also be used to create a versioned manifest in a specified environment without updating it.
@usingtmp
macro
This macro activates a temporary environment, optionally installs packages into it, and loads them with using
keyword.
- If current environment is already a temporary one, environment is not changed.
- If current env was a project (not package!), a temporary env will be activated.
- If current env was a package (under development), e.g.
MyPkg
, a temporary env will be activated, ANDMyPkg
will be dev-ed in that temporary env.
Thу last one is actually the interesting case, as you can continue to work on MyPkg
, while temporarily having additional packages available.
If @usingtmp
was called with arguments, the corresponding packages will be installed into that temporary env, and imported with using
keyword.
using ShareAdd
@usingtmp Foo, Bar
@usingtmp Baz: quux
Some other functions and usage cases
The macro @showenv
(with or without arguments) will open an environment folder in your desktop GUI, saving you from a bit of hassle of getting to hidden folder.
The functions ShareAdd.info()
, ShareAdd.update()
, ShareAdd.delete()
do what their names say.
The function ShareAdd.make_importable
also does what it says. It is used internally by @usingany
, but it can also be used separately in special cases, e.g. if you need using A as B
syntax, or want to import a package via import
statement instead of using
:
using ShareAdd: make_importable
make_importable("Foo")
import Foo
Workflow for upgrading Julia or moving to a different computer.
Upgrading Julia
Each Julia minor version has it's main shared environment in the correspondingly named folder, e.g. for Julia v1.11 the folder name is v1.11
, for the next version it will be v1.12
. All other shared environments are commonly used by all Julia versions.
For an upgrade e.g. from Julia v1.11 to v1.12 you can proceed as following:
- Before the very first run of Julia v1.12, make a copy of
v1.11
folder, and name itv1.12
. You can use@showenv
macro without arguments to open the environments folder in your desktop GUI.- Then, upon upgrade to Julia v1.12, update first the new main environment from the
Pkg
command line, - then update all shared environments with the help of
ShareAdd
.ShareAdd.update()
will create version-specific manifests, thus ensuring that you can use the same shared env with different versions of Julia without conflicts:
- Then, upon upgrade to Julia v1.12, update first the new main environment from the
(SomeEnv) pkg> activate # calling activate without arguments will activate the main env
Activating project at `~/.julia/environments/v1.12`
(@v1.12) pkg> update
# update info
julia> using ShareAdd
julia> ShareAdd.update()
# a lot of update infos
- Alternatively, if you have already run Julia v1.12 and thus the
v1.12
folder has already been created,- add
ShareAdd
to your main environment, - update all shared environments using
ShareAdd.update()
.
- add
Moving to a different computer
The procedure is in parts similar to described above for Julia upgrade.
- Open the environments folder e.g. by calling
@showenv
. - Copy all folders you want to transfer,
- and paste them into the environments folder on the new computer.
- If the main env folder (e.g.
v1.11
) was among the copied,ShareAdd
is assumed to be installed, otherwise install it there. - Update the main environment from the
Pkg
command line. - Update all shared environments using
ShareAdd.update()
.
Reference
Exported macros
ShareAdd.@showenv
— Macro@showenv
@showenv item
Open (shared) environment folder in your desktop GUI. The utility of this macro is that it makes it easy to access these folders in your OS, which might otherwise require some jumping through hoops, as these are located in the hidden folder ~/.julia/
This macro is exported.
Examples
julia> @showenv # called without arguments, opens the "environments" folder which contains all shared environments
julia> @showenv Revise # open the environment folder(s) which contain the Revise package
julia> @showenv "Revise" # both quoted and unquoted forms of the argument are OK
julia> @showenv Math # opens the folder of the shared env @Math
ShareAdd.@usingany
— Macro@usingany pkg
@usingany pkg1, pkg2, ...
@usingany pkg: fn
@usingany pkg: fn, @mcr, ...
@usingany kwarg = true [pkg...]
Makes package(s) available, if they are not already, and loads them with using
keyword.
- If a package is available in an environment in
LOAD_PATH
, that's OK. - If a package is available in a shared environment, this environment will be pushed into
LOAD_PATH
. - Otherwise if package(s) can be installed, you will be prompted to select an environment to install each package.
- If the package is not listed in any registry, an error will be thrown.
The macro can be called with keyword arguments:
update_pkg::Bool
: if set totrue
, first updates the package(s) to be imported by the macroupdate_env::Bool
: first update the shared environments currently in theLOAD_PATH
update_all::Bool
: first update ALL shared environments as well as the current project
If Julia version supports versioned manifests, on any updates a versioned manifest will be created in each updated env. See also make_current_mnf
and update
.
If update_all
or update_env
kwarg is set, @usingany
can be called without specifying any package(s) for import. If update_pkg
kwarg is set, package(s) to import must be specified.
This macro is exported.
Examples
julia> @usingany Foo, Bar
julia> @usingany Baz: quux
julia> @usingany update_all = true
julia> @usingany update_pkg = true Qux
ShareAdd.@usingtmp
— Macro@usingtmp
@usingtmp pkg
@usingtmp pkg1, pkg2, ...
@usingtmp pkg: fn
@usingtmp pkg: fn, @mcr, ...
Activates a temporary environment, optionally installs packages into it and loads them with using
keyword.
- If current environment is a temporary one, environment is not changed.
- If current env was a project (not package!), a temporary env will be activated.
- If current env was a package (under development), e.g.
MyPkg
, a temporary env will be activated, ANDMyPkg
will bedev
-ed in that temporary env.
Afterwards, if @usingtmp
was called with arguments, the corresponding packages will be installed into that temporary env, and imported with using
keyword.
This macro is exported.
Public functions
ShareAdd.current_env
— Methodcurrent_env(; depot = first(DEPOT_PATH)) -> EnvInfo
Returns information about the current active environment as an EnvInfo
object.
This function is public, not exported.
ShareAdd.delete
— Methoddelete(nms::Union{String, Vector{String}}; inall=ASKING, force = ASKING) -> nothing
delete(nm::AbstractString; inall=ASKING, force = ASKING) -> nothing
delete(p::Pair{<:AbstractString, <:AbstractString}; force = ASKING) -> nothing
Deletes shared envs, or packages therein.
If the provided argument is name(s) of shared environment(s), as specified by leading "@" in the names(s): then deletes the shared environment(s) by erasing their directories.
Otherwise, the provided name(s) are considered package names: then for each package pkg
deletes it from it's shared environment(s). Afterwards deletes the environment if it left empty after package deletion.
You can also specify both the env and the package in the form "@Foo" => "bar"
Keyword arguments
Both kwargs accept any integer types, including Bool, as well as enum SkipAskForceEnum
with integer values [-1=>SKIPPING, 0=>ASKING, 1=>FORCING]. If Bool is used, false
is equivalent to ASKING
, and true
to FORCING
inall=ASKING
: If set toFORCING
, would delete package from multiple environments, whereas withSKIPPING
, it will skip without asking. Has no effect if providednms
is/are env name(s).force=ASKING
: If set toFORCING
, would delete the env even if the env is currently inLOAD_PATH
, and remove a package even if it is loaded.
Examples
julia> ShareAdd.delete("@TrialPackages")
julia> ShareAdd.delete(["UnusedPkg", "UselessPkg"]; inall=true)
julia> ShareAdd.delete("@Foo" => "bar")
This function is public, not exported.
ShareAdd.info
— Methodinfo(nms::Union{Nothing, String, Vector{String}} = nothing;
by_env=true, listing=nothing, std_lib=false, upgradable=false, disp_rslt=true, ret_rslt=false)
Prints out and/or returns information about shared environments.
Argument
nms
: Name(s) of package(s) or environment(s) to return the information on. Environment names must start with "@". Package and env names cannot be provided together in one array.
Keyword arguments
by_env=true
: whether to print out results as aDict
of pairs like@env => [pkg1, ...]
, orpkg => [@env1, ...]
. Has no effect on returned (if any) results.listing=nothing
: this kwarg can benothing
,:envs
, or:pkgs
. If one of these twoSymbol
s is provided, the result is printed as a vector of envs or pkgs, resp. In this caseby_env
is ignored. Has no effect on returned (if any) resultsupgradable=false
: if true, all other kwargs will be ignored, and only upgradable packages with installed vs. most recent versions will be printed, ordered by environment.disp_rslt=true
: whether to print out results.ret_rslt=false
: whether the function returns anything. If set totrue
, it returns a NamedTuple(; env_dict, pkg_dict, envs, pkgs, absent)
, where the two first elements areDict
s with keywords correspondingly by env or by pkg;envs
andpkgs
are vectors of respective elements, andabsent
are those names provided through thenms
argument, which are not contained in the shared envs. Names of envs in the returned data are without leading "@".
This function is public, but not exported, as to avoid possible name conflicts.
Examples
julia> ShareAdd.info(["BenchmarkTools", "Chairmarks"])
The following packages are not in any shared env:
["Chairmarks"]
Found pkgs/envs:
@BenchmarkTools
=> ["BenchmarkTools"]
@Tools
=> ["BenchmarkTools"]
julia> ShareAdd.info(["DataFrames", "CSV"]; by_env=false)
CSV
=> ["@DataFrames"]
DataFrames
=> ["@DataFrames"]
julia> ShareAdd.info("StaticArrays"; upgradable=true)
@StaticArrays
StaticArrays: 1.9.8 --> 1.9.10
ShareAdd.make_current_mnf
— Methodmake_current_mnf(path_or_name) -> path
make_current_mnf(; current::Bool) -> path
make_current_mnf(env::EnvInfo) -> path
Creates a versioned manifest
If called make_current_mnf(; current=true)
, the current environment will be processed by this function.
path_or_name
can name of a shared environment starting with @
, or a path to any environment.
- If currently executed Julia version doesn't support version-specific manifests, do nothing.
- Else, if a versioned manifest for current Julia already exists, do nothing.
- Else, if the environment is the main shared env for the current Julia version (e.g. "@v1.11" for Julia v1.11), do nothing.
- Else, is a (versioned) manifest for an older Julia exists in the given directory, copy it to a file named according to the current Julia version, e.g.
Manifest-v1.11.toml
. - Else, create empty one.
Returns path to the created or existing manifest.
This function is public, not exported.
ShareAdd.make_importable
— Methodmake_importable(pkg::AbstractString)
make_importable(pkgs::AbstractVector{<:AbstractString})
make_importable(pkg1, pkg2, ...)
make_importable(::Nothing) => :success
Checks packages (by name only, UUIDs not supported!), prompts to install packages which are not in any shared environment, and adds relevant shared environments to LOAD_PATH
.
make_importable
is used internally by @usingany
, but it can be used separately e.g. if you e.g. want to import a package via import
statement instead of using
.
Returns :success
if the operation was successful, and nothing
if the user selected "Quit. Do Nothing." on any of the prompts.
Throws an error on unavailable packages.
Examples
julia> using ShareAdd
julia> make_importable("Foo")
:success
julia> import Foo
julia> using ShareAdd
julia> make_importable("Foo")
:success
julia> using Foo: bazaar as baz # @usingany Foo: bazaar as baz is not a supported syntax
This function is public, not exported.
ShareAdd.reset
— Methodreset()
Resets the LOAD_PATH
to it's default value of ["@", "@v#.#", "@stdlib"]
, thus removing any manually added paths.
This function is public, not exported.
ShareAdd.sh_add
— Methodsh_add(env_name::AbstractString; depot = first(DEPOT_PATH)) -> Vector{String}
sh_add(env_names::AbstractVector{<:AbstractString}; depot = first(DEPOT_PATH)) -> Vector{String}
sh_add(env_name::AbstractString, ARGS...; depot = first(DEPOT_PATH)) -> Vector{String}
Adds shared environment(s) to LOAD_PATH
, making the corresponding packages all available in the current session.
Returns the list of all packages in the added environments as a Vector{String}
.
Examples
julia> using ShareAdd: sh_add
julia> sh_add("@StatPackages")
3-element Vector{String}:
"Arrow"
"CSV"
"DataFrames"
julia> sh_add(["@StatPackages", "@Makie"])
4-element Vector{String}:
"Arrow"
"CSV"
"DataFrames"
"Makie"
julia> sh_add("@StatPackages", "@Makie")
4-element Vector{String}:
"Arrow"
"CSV"
"DataFrames"
"Makie"
This function is public, not exported.
ShareAdd.update
— Functionupdate()
update(nm::AbstractString; warn_if_missing=false)
update(nm::Vector{<:AbstractString}; warn_if_missing=true)
update(env::AbstractString, pkgs::Union{AbstractString, Vector{<:AbstractString}}; warn_if_missing=true)
update(env::EnvInfo, pkgs::Union{Nothing, S, Vector{S}} = Nothing; warn_if_missing=false) where S <: AbstractString
update(p::Pair{<:AbstractString, <:AbstractString}; warn_if_missing=false)
- Called without arguments, updates all shared environments.
- Called with a single argument
nm::String
starting with "@", updates the shared environmentnm
. - Called with a single argument
nm::String
not starting with "@", updates the packagenm
in all shared environments containing the package, as well as all it's downstream dependencies. - Called with a single argument
nm::Vector{String}
, updates the packages and/or environments innm
. - Called with two arguments
env
andpkgs
, updates the package(s)pkgs
in the environmentenv
. - Called with an argument
env => pkg
, updates the packagepkg
in the environmentenv
.
kwarg
warn_if_missing::Bool=true
- if env or pkg not found, issues a warning, otherwise would throw an error
If Julia version supports version-specific manifest, then on any updates a versioned manifest will be created in each updated env. See also make_current_mnf
.
Returnes nothing
.
Examples
julia> ShareAdd.update("@StatPackages")
julia> ShareAdd.update("@Foo" => "bar")
This function is public, not exported.
Public types
ShareAdd.EnvInfo
— Typemutable struct EnvInfo
EnvInfo(name::AbstractString) -> EnvInfo
name::String
- name of the environmentpath::String
- path of the environment's folderpkgs::Set{String}
- list of packages in the environmentin_path::Bool
- whether the environment is inLOAD_PATH
standard_env::Bool = false
- if the env is the standard one (which is in thev1.11
folder for Julia v1.11)shared::Bool = true
- if shared envtemporary::Bool = false
- if temporary envactive_project::Bool = false
- if active project
Examples
julia> ShareAdd.EnvInfo("@DocumenterTools")
ShareAdd.EnvInfo("DocumenterTools", "/Users/eben60/.julia/environments/DocumenterTools", Set(["DocumenterTools"]), false, false, true, false, false)
This type is public, not exported.
ShareAdd.PackageInfo
— Typemutable struct PackageInfo
name::String
- name of the packageenvs::Vector{EnvInfo}
- list of environments in which the package is presentin_path::Bool
- whether any of the environments is inLOAD_PATH
This type is public, not exported.
ShareAdd.SkipAskForceEnum
— TypeSkipAskForceEnum
Options for deletion of environments and packages.
SKIPPING = -1
ASKING = 0
FORCING = 1
As ShareAdd
is intended for interactive usage, and therefore the exported bindings are available in the Main
module, we use the "-ing" form to reduce the probability of name collisions.
The values of this enum are exported, whereas the enum type is public but not exported.