Changelog#
All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
v0.17.2#
Fixes#
Detection of CPU count in
define_clonotype_clusters
was broken (#527)
v0.17.1#
Fixes#
Compatibility with numpy 2.0 (#525)
Chore#
scverse template update to v0.4 (#519)
v0.17.0#
Additions#
Add “TCRdist” as new metric (#502)
Fixes#
Fix issue with detecting the number of available CPUs on MacOD (#518)
v0.16.1#
Fixes#
Fix default value for
n_jobs
inir.tl.ir_query
that could lead to an error (#498).Update description of D50 diversity metric in documentation (#499).
Fix
clonotype_modularity
not being able to store result in MuData in some cases (#504).Fix issue with creating sparse matrices from generators with the latest scipy version (#504)
v0.16.0#
Backwards-incompatible changes#
Use the
umi_count
field instead ofduplicate_count
to store UMI counts. The fieldumi_count
has been added to the AIRR Rearrangement standard in version 1.4 (#487). Use ofduplicate_count
for UMI counts is now discouraged. Scirpy will useumi_count
in allscirpy.io
functions. It will not change AIRR data that is read throughscirpy.io.read_airr
that still uses theduplicate_count
column. Scirpy remains compatible with datasets that still useduplicate_count
. You can update your dataset usingadata.obsm["airr"]["umi_count"] = adata.obsm["airr"]["duplicate_count"]
Other#
the
io.to_dandelion
andio.from_dandelion
interoperability functions now rely on the implementation provided by Dandelion itself (#483).
v0.15.0#
Fixes#
New features#
Speed up alignment distances by pre-filtering. There are two filtering strategies: A (lossless) length-based filter and a heuristic based on the expected penalty per mismatch. This is implemented in the
FastAlignmentDistanceCalculator
class which supersedes theAlignmentDistanceCalculator
class, which is now deprecated. Using the"alignment"
metric inpp.ir_dist
now uses theFastAlignmentDistanceCalculator
with only the lenght-based filter activated. Using the"fastalignment"
activates the heuristic, which is significantly faster, but results in some false-negatives. (#456)Switch to joblib/loky as a backend for parallel processing in
pp.ir_dist
. Joblib enables to switch to alternative backends that support out-of-machine computing (e.g.dask
,ray
) via theparallel_config
context manager. Additionally, chunk sizes are now adjusted dynamically based on the problem size. (#473)
Documentation#
The default values of the distance calculator classes in
ir_dist.metrics
was unclear. The default value is now set in the classes. Inpp.ir_dist
andir_dist.sequence_dist
, no cutoff argument is passed to the metrics objects, unless one is explicitly specified (previouslyNone
was passed by default).
v0.14.0#
Breaking changes#
Reimplement
pp.index_chains
using numba and awkward array functions, achieving a significant speedup. This function behaves exactly like the previous version except that callback functions passed to thefilter
arguments must now be vectorized over an awkward array, e.g. to check if ajunction_aa
field is present you could previously passlambda x: x['junction_aa'] is not None
, now an accepted version would belambda x: ~ak.is_none(x["junction_aa"], axis=-1)
. To learn more about native awkward array functions, please refer to the awkward array documentation. (#444)
Additions#
The
clonal_expansion
function now supports abreakpoints
argument for more flexible “expansion categories”. Thebreakpoints
argument supersedes theclip_at
parameter, which is now deprecated. (#439)
Fixes#
v0.13.1#
Fixes#
v0.13.0 - new data structure based on awkward arrays#
This update introduces a new datastructure based on awkward arrays. The new datastructure is described in more detail in the documentation and is considered the “official” way of representing AIRR data for scverse core and ecosystem packages.
Benefits of the new data structure include:
a more natural, lossless representation of AIRR Rearrangement data
separation of AIRR data and the receptor model, thereby getting rid of previous limitations (e.g. “only productive chains”) and enabling other use-cases (e.g. spatial AIRR data) in the future.
clean
adata.obs
as AIRR data is not expanded into columnssupport for MuData for working with paired gene expression and AIRR data as separate modalities.
The overall workflow stays the same, however this update required several backwards-incompatible changes which are summarized below.
Backwards-incompatible changes#
New data structure#
Closes issue https://github.com/scverse/scirpy/issues/327.
Changed behavior:
there are no “has_ir” and “multichain” columns in
adata.obs
anymoreBy default all fields are imported from AIRR rearrangement and 10x data.
The restriction that all chains added to an
AirrCell
must have the same fields has been removed. Missing fields are automatically filled with missing values.io.upgrade_schema
can update from v0.7 to v0.13 schema. AnnData objects generated with scirpy<= 0.6.x
cannot be read anymore.pl.spectratype
now has achain
attributed and the meaning of thecdr3_col
attribute has changed.
New functions:
pp.index_chains
pp.merge_chains
Removed functions:
pp.merge_with_ir
pp.merge_airr_chains
API supporting MuData#
Closes issue https://github.com/scverse/scirpy/issues/383
All functions take (where applicable) the additional, optional keyword arguments
airr_mod
: the modality in MuData that contains AIRR information (default: “airr”)airr_key
: the slot inadata.obsm
that contains AIRR rearrangement data (default: “airr”)chain_idx_key
: the slot inadata.obsm
that contains indices specifying which chains inadata.obsm[airr_key]
are the primary/secondary chains etc.
New class:
util.DataHandler
Updated example datasets#
The example datasets have been updated to be based on the new datastructure and are now based on MuData.
The example datasets have been regenerated from scratch using the loader notebooks described in the docstring. The Maynard dataset gene expression is now based on values generated with Salmon instead of RSEM/featurecounts.
Scirpy now uses pooch to manage example datasets.
Cleanup#
Removed the deprecated functions
io.from_tcr_objs
,io.from_ir_objs
,io.to_ir_objs
,pp.merge_with_tcr
,pp.tcr_neighbors
,pp.ir_neighbors
,tl.chain_pairing
Removed the deprecated classes
TcrCell
,AirrChain
,TcrChain
Removed the function
pl.cdr_convergence
which was never public anyway.
Additions#
Easy-access functions (scirpy.get
)#
Closes issue https://github.com/scverse/scirpy/issues/184
New functions:
get.airr
get.obs_context
get.airr_context
Fixes#
Several type hints that were previously inaccurate are now updated.
Fix x-axis labelling in
pl.clonotype_overlap
raises an error if row annotations are not unique for each group.
Documentation#
The documentation has been updated to reflect the changes described above, in particular the tutorials and the page about the data structure.
Moreover, the documentation now uses a new design and moved from GitHub pages to ReadTheDocs.org. Older versions of the documentation are still accessible from github pages.
Other changes#
Scirpy now adopts the cookiecutter-scverse template. The structure of this repository has ben adapted accordingly. Also code was reformatted in accordance with the template defaults.
The minimum required Python version is now 3.9 in accordance with NEP 29
Increased the minium version of tqdm to 4.63 (See https://github.com/tqdm/tqdm/issues/1082)
pl.repertoire_overlap
now always runstl.repertoire_overlap
internally and doesn’t rely on cached values.The mode
dendro_only
inpl.repertoire_overlap
has been removed.Cells that have a receptor, but no CDR3 sequence have previously received a separate clonotype in
tl.define_clonotypes
. Now they are receiving no clonotype (i.e.np.nan
) as do cells without a receptor.The function
tl.clonal_expansion
now returns apd.Series
instead of anp.array
withinplace=False
Removed deprecation for
clonotype_imbalanced
, see https://github.com/scverse/scirpy/issues/330The
group_abundance
tool and plotting function usedhas_ir
as a default group as we could previously rely on this column being present. With the new datastructure, this is not the case. To no break old code, thehas_ir
column is tempoarily added when requested. Thegroup_abundance
function will have to be rewritten enitrely in the future, see https://github.com/scverse/scirpy/issues/232In
pl.spectratype
, the parametergroupby
has been replaced bychain
.We now use isort to organize imports.
Static typing has been improved internally (using pylance). It’s not perfectly consistent yet, but we will keep working on this in the future.
Fix IEDB data loader after update of IEDB data formats (#401) and add tests for database import functions.
io.read_airr
now tolerates if fields required according to the AIRR standard are missing. The respective fields will be initalized withNone
(#407 by @zktuong).
v0.12.2#
Fixes#
Fix IEDB data loader after update of IEDB data formats (backport of #401)
v0.12.1#
Fixes#
Bump min Python version to 3.8; CI update by @grst in https://github.com/scverse/scirpy/pull/381
Temporarily pin pandas < 2 in #390
Other Changes#
update pre-commit CI
v0.12.0#
New Features#
Download IEDB and process it into an AnnData object by @ausserh in https://github.com/scverse/scirpy/pull/377
Fixes#
Fix working with subplots (#378) by @grst in https://github.com/scverse/scirpy/pull/379
Documentation#
Fix typos in IR query by @Zethson in https://github.com/scverse/scirpy/pull/374
Fix a bunch of typos in the docs by @grst in https://github.com/scverse/scirpy/pull/375
Internal changes#
Fix CI by @grst in https://github.com/scverse/scirpy/pull/376
New Contributors#
@Zethson made their first contribution in https://github.com/scverse/scirpy/pull/374
@ausserh made their first contribution in https://github.com/scverse/scirpy/pull/377
Full Changelog: https://github.com/scverse/scirpy/compare/v0.11.2…v0.12.0
v0.11.2#
Fixes#
Excluded broken python-igraph version (#366)
v0.11.1#
Fixes#
Solve incompatibility with scipy v1.9.0 (#360)
Internal changes#
do not autodeploy docs via CI (currently broken)
updated patched version of scikit-learn
v0.11.0#
Additions#
Add data loader for BD Rhapsody single-cell immune-cell receptor data (
io.read_bd_rhapsody
) (#351)
Fixes#
Fix type conversions in
from_dandelion
(#349).Update minimal dandelion version
Documentation#
Rebranding to scverse (#324, #326)
Add issue templates
Fix IMGT typos (#344 by @emjbishop)
Internal changes#
Bump default CI python version to 3.9
Use patched version of scikit-bio in CI until https://github.com/biocore/scikit-bio/pull/1813 gets merged
v0.10.1#
Fixes#
Fix bug in cellranger import (#310 by @ddemaeyer)
Fix that VDJDB download failed when cache dir was not present (#311)
v0.10.0#
Additions#
This release adds a new feature to query reference databases (#298) comprising
an extension of
pp.ir_dist
to compute distances to a reference dataset,tl.ir_query
, to match immune receptors to a reference database based on the distances computed withir_dist
,tl.ir_query_annotate
andtl.ir_query_annotate_df
to annotate cells based on the result oftl.ir_query
, anddatasets.vdjdb
which conveniently downloads and processes the latest version of VDJDB.
Fixes#
Bump minimal dependencies for networkx and tqdm (#300)
Fix issue with
repertoire_overlap
(Fix #302 via #305)Fix issue with
define_clonotype_clusters
(Fix #303 via #305)Suppress
FutureWarning
s from pandas in tutorials (#307)
Internal changes#
Update sphinx to >= 4.1 (#306)
Update black version
Update the internal folder structure:
tl
,pp
etc. are now real packages instead of aliases
v0.9.1#
Fixes#
Scirpy can now import additional columns from Cellranger 6 (#279 by @naity)
Fix minor issue with
include_fields
inAirrCell
(#297)
Documentation#
Fix broken link in README (#296)
Add developer documentation (#294)
v0.9.0#
Additions#
Add the new “clonotype modularity” tool which ranks clonotypes by how strongly connected their gene expression neighborhood graph is. (#282).
The below example shows three clonotypes (164, 1363, 942), two of which consist of cells that are transcriptionally related.
example clonotypes | clonotype modularity vs. FDR |
---|---|
Deprecations#
tl.clonotype_imbalance
is now deprecated in favor of the new clonotype modularity tool.
Fixes#
Fix calling locus from gene name in some cases (#288)
Compatibility with
networkx>=2.6
(#292)
Minor updates#
Fix some links in README (#284)
Fix old instances of clonotype in docs (should be clone_id) (#287)
v0.8.0#
Additions#
tl.alpha_diversity
now supports all metrics from scikit-bio, theD50
metric and custom callback functions (#277 by @naity)
Fixes#
Handle input data with “productive” chains which don’t have a
junction_aa
sequence annotated (#281)Fix issue with serialized “extra chains” not being imported correctly (#283 by @zktuong)
Minor changes#
The CI can now build documentation from pull-requests from forks. PR docs are not deployed to github-pages anymore, but can be downloaded as artifact from the CI run.
v0.7.1#
Fixes#
Ensure Compatibility with latest version of dandelion (e78701c)
Add links to older versions of documentation (#275)
Fix issue, where clonotype analysis couldn’t be continued after saving and reloading
h5ad
object (#274)Allow “None” values to be present as cell-level attributes during
merge_airr_chains
(#273)
Minor changes#
Require
anndata >= 0.7.6
in conda tests (#266)
v0.7.0#
This update features a
change of Scirpy’s data structure to improve interoperability with the AIRR standard
a complete re-write of the clonotype definition module for improved performance.
This required several backwards-incompatible changes. Please read the release notes below and the updated tutorials.
Backwards-incompatible changes#
Improve Interoperability by fully supporting the AIRR standard (#241)#
Scirpy stores receptor information in adata.obs
. In this release, we updated the column names to match the AIRR Rearrangement standard. Our data model is now much more flexible, allowing to import arbitrary immune-receptor (IR)-chain related information. Use scirpy.io.upgrade_schema()
to update existing AnnData
objects to the latest format.
Closed issues #240, #253, #258, #255, #242, #215.
This update includes the following changes:
IrCell
is now replaced byAirrCell
which has additional functionalityIrChain
has been removed. Use a plain dictionary instead.CDR3 information is now read from the
junction
andjunction_aa
columns instead ofcdr3_nt
andcdr3
, respectively.Clonotype assignments are now per default stored in the
clone_id
column.expr
andexpr_raw
are nowduplicate_count
andconsensus_count
.{v,d,j,c}_gene
is now{v,d,j,c}_call
.There’s now an
extra_chains
column containing all IR-chains that don’t fit into our receptor model. These chains are not used by scirpy, but can be re-exported to different formats.merge_with_ir
is now split up intomerge_with_ir
(to merge IR data with transcriptomics data) andmerge_airr_chains
(to merge several adatas with IR information, e.g. BCR and TCR data).Tutorial and documentation updates, to reflect these changes
Sequences are not converted to upper case on import. Scirpy tools that consume the sequences convert them to upper case on-the-fly.
{to,from}_ir_objs
has been renamed to{to,from}_airr_cells
.
Refactor CDR3 network creation (#230)#
Previously, pp.ir_neighbors
constructed a cell x cell
network based on clonotype similarity. This led to performance issues
with highly expanded clonotypes (i.e. thousands of cells with exactly the same receptor configuration). Such cells would
form dense blocks in the sparse adjacency matrix (see issue #217). Another downside was that expensive alignment-distances had
to be recomputed every time the parameters of ir_neighbors
was changed.
The new implementation computes distances between all unique receptor configurations, only considering one instance of highly expanded clonotypes.
Closed issues #243, #217, #191, #192, #164.
This update includes the following changes:
pp.ir_neighbors
has been replaced bypp.ir_dist
.The options
receptor_arms
anddual_ir
have been moved frompp.ir_neighbors
totl.define_clonotypes
andtl.define_clonotype_clusters
.The default key for clonotype clusters is now
cc_{distance}_{metric}
instead ofct_cluster_{distance}_{metric}
.same_v_gene
now fully respects the optionsdual_ir
andreceptor_arms
v-genes and receptor types were previously simply appended to clonotype ids (when
same_v_gene=True
). Now clonotypes with different v-genes get assigned a different numeric id.Distance metric classes have been moved from
ir_dist
toir_dist.metrics
.Distances matrices generated by
ir_dist
are now square and symmetric instead of triangular.The default value for
dual_ir
is nowany
instead ofprimary_only
(Closes #164).The API of
clonotype_network
has changed.Clonotype network now visualizes cells with identical receptor configurations. The number of cells with identical receptor configurations is shown as point size (and optionally, as color). Clonotype network does not support plotting multiple colors at the same time any more.
Clonotype network (previous implementation) |
Clonotype network (now) |
---|---|
Each dot represents a cell. Cells with identical receptors form a fully connected subnetwork |
Each dot represents cells with identical receptors. The dot size refers to the number of cells |
Drop Support for Python 3.6#
Support Python 3.9, drop support for Python 3.6, following the numpy guidelines. (#229)
Fixes#
tl.clonal_expansion
andtl.clonotype_convergence
now respect cells with missing receptors and returnnan
for those cells. (#252)
Additions#
util.graph.igraph_from_sparse_matrix
allows to convert a sparse connectivity or distance matrix to anigraph
object.ir_dist.sequence_dist
now also works sequence arrays that contain duplicate entries (#192)from_dandelion
andto_dandelion
facilitate interaction with the Dandelion package (#240)write_airr
allows to write scirpy’sadata.obs
back to the AIRR Rearrangement format.read_airr
now tries to infer the locus from gene names, if no locus column is present.ir.io.upgrade_schema
allows to upgrade an existing scirpy anndata object to be compatible with the latest version of scirpydefine_clonotypes
anddefine_clonotype_clusters
now prints a logging message indicating where the results have been stored (#215)
Minor changes#
tqdm
now uses IPython widgets to display progress bars, if availablethe
process_map
fromtqdm
is now used to display progress bars for parallel computations instead the custom implementation used previously f307c2bmatplotlib
s “grid lines” are now suppressed by default in all plots.Docs from the
master
branch are now deployed toicbi-lab.github.io/scirpy/develop
instead of the main documentation website. The main website only gets updated on releases.Refactored the
_is_na
function that checks if a string evaluates toNone
.Fixed outdated documentation of the
receptor_arms
parameter (#264)
v0.6.1#
Fixes#
Fix an issue where
define_clonotype
failed when the clonotype network had no edges (#236).Require pandas >= 1.0 and fix a pandas incompatibility in
merge_with_ir
(#238).Ensure consistent order of the spectratype dataframe (#238).
Minor changes#
Fix missing
bibtex_bibfiles
option in sphinx configurationWork around https://github.com/takluyver/flit/issues/383.
v0.6.0#
Backwards-incompatible changes:#
Set more sensible defaults the the
cutoff
parameter inir_neighbors
. The default is now2
forhamming
andlevenshtein
distance metrics and10
for thealignment
distance metric.
Additions:#
Add Hamming-distance as additional distance metric for
ir_neighbors
(#216 by @ktpolanski)
Minor changes:#
Fix MacOS CI (#221)
Use mamba instead of conda in CI (#216)
v0.5.0 - Add support for BCRs and gamma-delta TCRs#
Backwards-incompatible changes:#
The data structure has changed. Column have been renamed from
TRA_xxx
andTRB_xxx
toIR_VJ_xxx
andIR_VDJ_xxx
. Additionally alocus
column has been added for each chain.All occurences of
tcr
in the function and class names have been replaced withir
. Aliases for the old names have been created and emit aFutureWarning
.
Additions:#
There’s now a mixed TCR/BCR example dataset (
maynard2020
) available (#211)BCR-related amendments to the documentation (#206)
tl.chain_qc
which supersedeschain_pairing
. It additionally provides information about the receptor type.io.read_tracer
now supports gamma-delta T-cells (#207)io.to_ir_objs
allows to convert adata to a list ofIrCells
(#210)io.read_bracer
allows to read-in BraCeR BCR data. (#208)The
pp.merge_with_ir
function now can handle the case when both the left and the rightAnnData
object contain immune receptor information. This is useful when integrating both TCR and BCR data into the same dataset. (#210)
Fixes:#
Fix a bug in
vdj_usage
which has been triggered by the new data structure (#203)
Minor changes:#
Removed the tqdm monkey patch, as the issue has been resolved upstream (#200)
Add AIRR badge, as scirpy is now certified to comply with the AIRR software standard v1. (#202)
Require pycairo >1.20 which provides a windows wheel, eliminating the CI problems.
v0.4.2#
Include tests into main package (#189)
v0.4.1#
Fix pythonpublish CI action
Update black version (and code style, accordingly)
Changes for AIRR-complicance:
Add support level to README
Add Biocontainer instructions to README
Add a minimal test suite to be ran on conda CI
v0.4#
Adapt tcr_dist to support second array of sequences (#166). This enables comparing CDR3 sequences against a list of reference sequences.
Add
tl.clonotype_convergence
which helps to find evidence of convergent evolution (#168)Optimize parallel sequence distance calculation (#171). There is now less communication overhead with the worker processes.
Fixed an error when runing
pp.tcr_neighbors
(#177)Improve packaging. Use
setuptools_scm
instead ofget_version
. Remove redundant metadata. (#180). More tests for conda (#180).
v0.3#
More extensive CI tests (now also testing on Windows, MacOS and testing the conda recipe) (#136, #138)
Add example images to API documentation (#140)
Refactor IO to expose TcrCell and TcrChain (#139)
Create data loading tutorial (#139)
Add a progressbar to TCR neighbors (#143)
Move clonotype_network_igraph to tools (#144)
Add
read_airr
to support the AIRR rearrangement format (#147)Add option to take v-gene into account during clonotype definition (#148)
Store colors in AnnData to ensure consistent coloring across plots (#151)
Divide
define_clontoypes
intodefine_clonotypes
anddefine_clonotype_clusters
(#152). Now, the user has to specify explicitlysequence
andmetric
for bothtl.tcr_neighbors
,tl.define_clonotype_clusters
andtl.clonotype_network
. This makes it more straightforward to have multiple, different versions of the clonotype network at the same time. The default parameters changed tosequence="nt"
and `metric=”identity” to comply with the traditional definition of clonotypes. The changes are also reflected in the glossary and the tutorial.Update the workflow figure (#154)
Fix a bug that caused labels in the
repertoire_overlap
heatmap to be mixed up. (#157)Add a label to the heatmap annotation in
repertoire_overlap
(#158).
v0.2#
Documentation overhaul. A lot of docstrings got corrected and improved and the formatting of the documentation now matches scanpy’s.
Experimental function to assess bias in clonotype abundance between conditions (#92)
Scirpy now has a logo (#123)
Update default parameters for
clonotype_network
:Edges are now only automatically displayed if plotting < 1000 nodes
If plotting variables with many categories, the legend is hidden.
Update default parameters for alignment-based
tcr_neighbors
The gap extend penalty now equals the gap open penalty (
11
).
v0.1.2#
Make 10x csv and json import consistent (#109)
Fix version requirements (#112)
Fix compatibility issues with pandas > 1 (#112)
Updates to tutorial and README
v0.1.1#
Update documentation about T-cell receptor model (#4, #10)
Update README
Fix curve plots (#31)
Host datasets on GitHub (#104)
v0.1#
Initial release for pre-print