License: MIT Documentation Build Status Coverage Aqua QA

ShareAdd.jl

This Julia package helps to reduce clutter in your main shared environment (and thus avoid package incompatibility problems) by making it easy to use multiple shared or temporary environments. It exports two macros: @usingany and @usingtmp, envisioned for two different workflows. The package also provides several utility functions for managing shared environments.

Glossary

The definitions below deviate somewhat from the strict definitions of Environment, Project, and Package as given in the Julia docs, and refer to the most common and relevant cases.

  • Project File: The Project.toml file in environment's folder. Its deps section lists packages available in this env. May contain additional data like compatibility requirements. 🔗
  • Manifest: TOML-format file in environment's folder. It contains the actual state of the environment, including all indirect dependencies and their versions. Environment can contain multiple Manifest files for different Julia versions. 🔗
  • Environment: Simply a folder containing a Project.toml and optionally a Manifest file. Projects or packages are also environments. All packages in the Project.toml of the active environment are available for import. 🔗
  • Shared Environment: Any environment in the path-to-Julia/.Julia/environments folder. These environments can be addressed by prepending @ to the env name, without typing it's whole path. Typically shared environments do not contain projects. 🔗
  • Main / Default Environment: The subfolder in the environments folder, named according to the Julia minor version , e.g. for Julia v1.11 it's name will be v1.11. This is the default env upon start of Julia, except Julia was started with some specific project. Especially for novices it is common to work in the default environment and install all packages therein. This may result in compat conflicts. The motivation to create ShareAdd was to help avoiding this kind of situaltion.
  • LOAD_PATH: An array of currently available environments. The default value is ["@", "@v#.#", "@stdlib"], where "@" refers to the current project, and "@v#.#" stands for the "main" env. Environments can be added both in the form of env folder paths, or, for shared envs, as the env name prepended by @ - e.g. ["@", "@v#.#", "@stdlib", "@MyTools"].
  • Stacked Environments: A concept of using multiple environments at the same time, so that the packages from each environment are available. Realized by having multiple (shared) envs in the LOAD_PATH. This is the main mechanism behind ShareAdd. 🔗.
  • Project: A directory containing a Project.toml file as well as source and other relevant files. Project is a subtype of environment.
  • Package: Is a project adhering to a certain structure, which contains source files and other resources, suitable for distribution. Packages can be loaded with using or import. 🔗

@usingany macro

This macro makes package(s) available, if they are not already, and loads them with using keyword.

  • If a package is available in an environment in LOAD_PATH, that's OK.
  • If a package is available in a shared environment, this environment will be pushed into LOAD_PATH.
  • Otherwise if it can be installed, you will be prompted to select an environment to install the package(s).
  • If the package is not listed in any registry, an error will be thrown.
# simplest usage case
@usingany SomePackage

For more usage options see @usingany docs .

@usingany usage example

Let's assume, while working on your package MyPackage, we temporarily need packages TOML, Plots, and Chairmarks. However, they shouldn't be added permanently to your package dependencies. Furthermore, from the package BenchmarkTools we need only the macro @btime and the function save. We also need Unitful, which is already an installed dependence of MyPackage.

TOML is available in the stdlib, Plots and BenchmarkTools you already put into a shared environment @utilities, and Chairmarks is not on your computer yet.

Now, first, you add ShareAdd to your "main" (standard) enviroment, making it available at all times:

]
(YourEnv) pkg> activate 
  Activating project at `~/.julia/environments/v1.11`

(@v1.11) pkg> add ShareAdd
(...)
(@v1.11 pkg> activate . # back to your environment
(YourEnv) pkg> 

By that occasion you may also want to clean your standard environment: It is generally not recommended having a lot of packages there.

Now, the only thing you need is to type into REPL (or adding to your script) the following lines:

using ShareAdd
@usingany Unitful, TOML, Plots, Chairmarks
@usingany BenchmarkTools: @btime, save

As Chairmarks was not installed yet, you will be asked as to where to install it. You may e.g. add it to your existing @utilities shared environment, or let create a new environment @Chairmarks and put it there.

Afterwards @utilities (and @Chairmarks, if created) will be added to LOAD_PATH, making their packages available.

Finally, the macros will execute using Unitful, TOML, Plots, Chairmarks resp. using BenchmarkTools: @btime, save - and that's it. Enjoy!

@usingany with updates

By setting the corresponding kwarg, it is possible to first update the packages and/or environments prior to execution of import. E.g. the following command would update the packages Pkg1, Pkg2 in their shared environments:

using ShareAdd
@usingany update_pkg = true Pkg1, Pkg2

@usingany without explicitly calling @usingany

ShareAdd.jl can be combined nicely with BasicAutoloads.jl. See this Discourse post to learn how get packages silently loaded if you call their functions in the REPL - e.g. if you type mean([1,2,3]) or 1.55u"V".

Versioned manifests

If your currently used Julia version supports versioned manifests (i.e. >= v1.10.8), then on any updates using ShareAdd package (see ShareAdd.update), a versioned manifest will be created in each updated env. The function ShareAdd.make_current_mnf can also be used to create a versioned manifest in a specified environment without updating it.

@usingtmp macro

This macro activates a temporary environment, optionally installs packages into it, and loads them with using keyword.

  • If current environment is already a temporary one, environment is not changed.
  • If current env was a project (not package!), a temporary env will be activated.
  • If current env was a package (under development), e.g. MyPkg, a temporary env will be activated, AND MyPkg will be dev-ed in that temporary env.

Thу last one is actually the interesting case, as you can continue to work on MyPkg, while temporarily having additional packages available.

If @usingtmp was called with arguments, the corresponding packages will be installed into that temporary env, and imported with using keyword.

using ShareAdd
@usingtmp Foo, Bar
@usingtmp Baz: quux

Some other functions and usage cases

The macro @showenv (with or without arguments) will open an environment folder in your desktop GUI, saving you from a bit of hassle of getting to hidden folder.

The functions ShareAdd.info(), ShareAdd.update(), ShareAdd.delete() do what their names say.

The function ShareAdd.make_importable also does what it says. It is used internally by @usingany, but it can also be used separately in special cases, e.g. if you need using A as B syntax, or want to import a package via import statement instead of using:

using ShareAdd: make_importable
make_importable("Foo")
import Foo

Workflow for upgrading Julia or moving to a different computer.

Upgrading Julia

Each Julia minor version has it's main shared environment in the correspondingly named folder, e.g. for Julia v1.11 the folder name is v1.11, for the next version it will be v1.12. All other shared environments are commonly used by all Julia versions.

For an upgrade e.g. from Julia v1.11 to v1.12 you can proceed as following:

  • Before the very first run of Julia v1.12, make a copy of v1.11 folder, and name it v1.12. You can use @showenv macro without arguments to open the environments folder in your desktop GUI.
    • Then, upon upgrade to Julia v1.12, update first the new main environment from the Pkg command line,
    • then update all shared environments with the help of ShareAdd. ShareAdd.update() will create version-specific manifests, thus ensuring that you can use the same shared env with different versions of Julia without conflicts:
(SomeEnv) pkg> activate # calling activate without arguments will activate the main env
  Activating project at `~/.julia/environments/v1.12`

(@v1.12) pkg> update
# update info

julia> using ShareAdd

julia> ShareAdd.update()
# a lot of update infos
  • Alternatively, if you have already run Julia v1.12 and thus the v1.12 folder has already been created,
    • add ShareAdd to your main environment,
    • update all shared environments using ShareAdd.update().

Moving to a different computer

The procedure is in parts similar to described above for Julia upgrade.

  • Open the environments folder e.g. by calling @showenv.
  • Copy all folders you want to transfer,
  • and paste them into the environments folder on the new computer.
  • If the main env folder (e.g. v1.11) was among the copied, ShareAdd is assumed to be installed, otherwise install it there.
  • Update the main environment from the Pkg command line.
  • Update all shared environments using ShareAdd.update().

Reference

Exported macros

ShareAdd.@showenvMacro
@showenv
@showenv item

Open (shared) environment folder in your desktop GUI. The utility of this macro is that it makes it easy to access these folders in your OS, which might otherwise require some jumping through hoops, as these are located in the hidden folder ~/.julia/

This macro is exported.

Examples

julia> @showenv # called without arguments, opens the "environments" folder which contains all shared environments
julia> @showenv Revise # open the environment folder(s) which contain the Revise package
julia> @showenv "Revise" # both quoted and unquoted forms of the argument are OK
julia> @showenv Math # opens the folder of the shared env @Math
source
ShareAdd.@usinganyMacro
@usingany pkg
@usingany pkg1, pkg2, ... 
@usingany pkg: fn
@usingany pkg: fn, @mcr, ... 
@usingany kwarg = true [pkg...]

Makes package(s) available, if they are not already, and loads them with using keyword.

  • If a package is available in an environment in LOAD_PATH, that's OK.
  • If a package is available in a shared environment, this environment will be pushed into LOAD_PATH.
  • Otherwise if package(s) can be installed, you will be prompted to select an environment to install each package.
  • If the package is not listed in any registry, an error will be thrown.

The macro can be called with keyword arguments:

  • update_pkg::Bool: if set to true, first updates the package(s) to be imported by the macro
  • update_env::Bool: first update the shared environments currently in the LOAD_PATH
  • update_all::Bool: first update ALL shared environments as well as the current project

If Julia version supports versioned manifests, on any updates a versioned manifest will be created in each updated env. See also make_current_mnf and update.

If update_all or update_env kwarg is set, @usingany can be called without specifying any package(s) for import. If update_pkg kwarg is set, package(s) to import must be specified.

This macro is exported.

Examples

julia> @usingany Foo, Bar
julia> @usingany Baz: quux
julia> @usingany update_all = true
julia> @usingany update_pkg = true Qux
source
ShareAdd.@usingtmpMacro
@usingtmp 
@usingtmp pkg
@usingtmp pkg1, pkg2, ... 
@usingtmp pkg: fn
@usingtmp pkg: fn, @mcr, ...

Activates a temporary environment, optionally installs packages into it and loads them with using keyword.

  • If current environment is a temporary one, environment is not changed.
  • If current env was a project (not package!), a temporary env will be activated.
  • If current env was a package (under development), e.g. MyPkg, a temporary env will be activated, AND MyPkg will be dev-ed in that temporary env.

Afterwards, if @usingtmp was called with arguments, the corresponding packages will be installed into that temporary env, and imported with using keyword.

This macro is exported.

source

Public functions

ShareAdd.current_envMethod
current_env(; depot = first(DEPOT_PATH)) -> EnvInfo

Returns information about the current active environment as an EnvInfo object.

This function is public, not exported.

source
ShareAdd.deleteMethod
delete(nms::Union{String, Vector{String}}; inall=ASKING, force = ASKING) -> nothing
delete(nm::AbstractString; inall=ASKING, force = ASKING) -> nothing
delete(p::Pair{<:AbstractString, <:AbstractString}; force = ASKING) -> nothing

Deletes shared envs, or packages therein.

If the provided argument is name(s) of shared environment(s), as specified by leading "@" in the names(s): then deletes the shared environment(s) by erasing their directories.

Otherwise, the provided name(s) are considered package names: then for each package pkg deletes it from it's shared environment(s). Afterwards deletes the environment if it left empty after package deletion.

You can also specify both the env and the package in the form "@Foo" => "bar"

Keyword arguments

Both kwargs accept any integer types, including Bool, as well as enum SkipAskForceEnum with integer values [-1=>SKIPPING, 0=>ASKING, 1=>FORCING]. If Bool is used, false is equivalent to ASKING, and true to FORCING

  • inall=ASKING: If set to FORCING, would delete package from multiple environments, whereas with SKIPPING, it will skip without asking. Has no effect if provided nms is/are env name(s).
  • force=ASKING: If set to FORCING, would delete the env even if the env is currently in LOAD_PATH, and remove a package even if it is loaded.

Examples

julia> ShareAdd.delete("@TrialPackages")
julia> ShareAdd.delete(["UnusedPkg", "UselessPkg"]; inall=true)
julia> ShareAdd.delete("@Foo" => "bar")

This function is public, not exported.

source
ShareAdd.infoMethod
info(nms::Union{Nothing, String, Vector{String}} = nothing; 
    by_env=true, listing=nothing, std_lib=false, upgradable=false, disp_rslt=true, ret_rslt=false)

Prints out and/or returns information about shared environments.

Argument

  • nms: Name(s) of package(s) or environment(s) to return the information on. Environment names must start with "@". Package and env names cannot be provided together in one array.

Keyword arguments

  • by_env=true: whether to print out results as a Dict of pairs like @env => [pkg1, ...], or pkg => [@env1, ...]. Has no effect on returned (if any) results.
  • listing=nothing: this kwarg can be nothing, :envs, or :pkgs. If one of these two Symbols is provided, the result is printed as a vector of envs or pkgs, resp. In this case by_env is ignored. Has no effect on returned (if any) results
  • upgradable=false: if true, all other kwargs will be ignored, and only upgradable packages with installed vs. most recent versions will be printed, ordered by environment.
  • disp_rslt=true: whether to print out results.
  • ret_rslt=false: whether the function returns anything. If set to true, it returns a NamedTuple (; env_dict, pkg_dict, envs, pkgs, absent), where the two first elements are Dicts with keywords correspondingly by env or by pkg; envs and pkgs are vectors of respective elements, and absent are those names provided through the nms argument, which are not contained in the shared envs. Names of envs in the returned data are without leading "@".

This function is public, but not exported, as to avoid possible name conflicts.

Examples

julia> ShareAdd.info(["BenchmarkTools", "Chairmarks"])
The following packages are not in any shared env:
    ["Chairmarks"]

Found pkgs/envs:
  @BenchmarkTools
   => ["BenchmarkTools"]
  @Tools
   => ["BenchmarkTools"]

julia> ShareAdd.info(["DataFrames", "CSV"]; by_env=false)
  CSV
   => ["@DataFrames"]
  DataFrames
   => ["@DataFrames"]

julia> ShareAdd.info("StaticArrays"; upgradable=true)
  @StaticArrays
    StaticArrays: 1.9.8 --> 1.9.10   
source
ShareAdd.make_current_mnfMethod
make_current_mnf(path_or_name) -> path
make_current_mnf(; current::Bool) -> path
make_current_mnf(env::EnvInfo) -> path

Creates a versioned manifest

If called make_current_mnf(; current=true), the current environment will be processed by this function.

path_or_name can name of a shared environment starting with @, or a path to any environment.

  • If currently executed Julia version doesn't support version-specific manifests, do nothing.
  • Else, if a versioned manifest for current Julia already exists, do nothing.
  • Else, if the environment is the main shared env for the current Julia version (e.g. "@v1.11" for Julia v1.11), do nothing.
  • Else, is a (versioned) manifest for an older Julia exists in the given directory, copy it to a file named according to the current Julia version, e.g. Manifest-v1.11.toml.
  • Else, create empty one.

Returns path to the created or existing manifest.

This function is public, not exported.

source
ShareAdd.make_importableMethod
make_importable(pkg::AbstractString)
make_importable(pkgs::AbstractVector{<:AbstractString})
make_importable(pkg1, pkg2, ...)
make_importable(::Nothing) => :success

Checks packages (by name only, UUIDs not supported!), prompts to install packages which are not in any shared environment, and adds relevant shared environments to LOAD_PATH.

make_importable is used internally by @usingany, but it can be used separately e.g. if you e.g. want to import a package via import statement instead of using.

Returns :success if the operation was successful, and nothing if the user selected "Quit. Do Nothing." on any of the prompts.

Throws an error on unavailable packages.

Examples

julia> using ShareAdd
julia> make_importable("Foo")
:success
julia> import Foo 

julia> using ShareAdd
julia> make_importable("Foo")
:success
julia> using Foo: bazaar as baz  # @usingany Foo: bazaar as baz is not a supported syntax

This function is public, not exported.

source
ShareAdd.resetMethod
reset()

Resets the LOAD_PATH to it's default value of ["@", "@v#.#", "@stdlib"], thus removing any manually added paths.

This function is public, not exported.

source
ShareAdd.sh_addMethod
sh_add(env_name::AbstractString; depot = first(DEPOT_PATH)) -> Vector{String}
sh_add(env_names::AbstractVector{<:AbstractString}; depot = first(DEPOT_PATH)) -> Vector{String}
sh_add(env_name::AbstractString, ARGS...; depot = first(DEPOT_PATH)) -> Vector{String}

Adds shared environment(s) to LOAD_PATH, making the corresponding packages all available in the current session.

Returns the list of all packages in the added environments as a Vector{String}.

Examples

julia> using ShareAdd: sh_add
julia> sh_add("@StatPackages")
3-element Vector{String}:
 "Arrow"
 "CSV"
 "DataFrames"

julia> sh_add(["@StatPackages", "@Makie"])
4-element Vector{String}:
 "Arrow"
 "CSV"
 "DataFrames"
 "Makie"

julia> sh_add("@StatPackages", "@Makie")
4-element Vector{String}:
 "Arrow"
 "CSV"
 "DataFrames"
 "Makie"

This function is public, not exported.

source
ShareAdd.updateFunction
update()
update(nm::AbstractString; warn_if_missing=false)
update(nm::Vector{<:AbstractString}; warn_if_missing=true)
update(env::AbstractString, pkgs::Union{AbstractString, Vector{<:AbstractString}}; warn_if_missing=true) 
update(env::EnvInfo, pkgs::Union{Nothing, S, Vector{S}} = Nothing; warn_if_missing=false) where S <: AbstractString
update(p::Pair{<:AbstractString, <:AbstractString}; warn_if_missing=false)
  • Called without arguments, updates all shared environments.
  • Called with a single argument nm::String starting with "@", updates the shared environment nm.
  • Called with a single argument nm::String not starting with "@", updates the package nm in all shared environments containing the package, as well as all it's downstream dependencies.
  • Called with a single argument nm::Vector{String}, updates the packages and/or environments in nm.
  • Called with two arguments env and pkgs, updates the package(s) pkgs in the environment env.
  • Called with an argument env => pkg, updates the package pkg in the environment env.

kwarg

  • warn_if_missing::Bool=true - if env or pkg not found, issues a warning, otherwise would throw an error

If Julia version supports version-specific manifest, then on any updates a versioned manifest will be created in each updated env. See also make_current_mnf.

Returnes nothing.

Examples

julia> ShareAdd.update("@StatPackages")
julia> ShareAdd.update("@Foo" => "bar")

This function is public, not exported.

source

Public types

ShareAdd.EnvInfoType
mutable struct EnvInfo
EnvInfo(name::AbstractString) -> EnvInfo
  • name::String - name of the environment
  • path::String - path of the environment's folder
  • pkgs::Set{String} - list of packages in the environment
  • in_path::Bool - whether the environment is in LOAD_PATH
  • standard_env::Bool = false - if the env is the standard one (which is in the v1.11 folder for Julia v1.11)
  • shared::Bool = true - if shared env
  • temporary::Bool = false - if temporary env
  • active_project::Bool = false - if active project

Examples

julia> ShareAdd.EnvInfo("@DocumenterTools")
ShareAdd.EnvInfo("DocumenterTools", "/Users/eben60/.julia/environments/DocumenterTools", Set(["DocumenterTools"]), false, false, true, false, false)

This type is public, not exported.

source
ShareAdd.PackageInfoType
mutable struct PackageInfo
  • name::String - name of the package
  • envs::Vector{EnvInfo} - list of environments in which the package is present
  • in_path::Bool - whether any of the environments is in LOAD_PATH

This type is public, not exported.

source
ShareAdd.SkipAskForceEnumType
SkipAskForceEnum

Options for deletion of environments and packages.

  • SKIPPING = -1
  • ASKING = 0
  • FORCING = 1

As ShareAdd is intended for interactive usage, and therefore the exported bindings are available in the Main module, we use the "-ing" form to reduce the probability of name collisions.

The values of this enum are exported, whereas the enum type is public but not exported.

source