Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CytoNorm by Flowjo=> No normalized files created #37

Open
BaptLabRight opened this issue Dec 6, 2022 · 7 comments
Open

CytoNorm by Flowjo=> No normalized files created #37

BaptLabRight opened this issue Dec 6, 2022 · 7 comments

Comments

@BaptLabRight
Copy link

Hi,
I am a CytoNorm user via FlowJo software.

The normalization process seems to work but when it ends, I can't find any "nomalised" fcs files. In contrast, some new R scripts such as "RScript.CytoNorm.1670259468256.R" are present in the original folder containing the fcs files I used for normalisation.

Where can I find the normalized files? How can I be sure that no error occurred during the normailsation?
Thank you
@zlanzar
Copy link

zlanzar commented Jan 19, 2023

Hello,

I'm also running into the above error. Rscript output shows:

R version 4.1.1 (2021-08-10) -- "Kick Things"
Copyright (C) 2021 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin17.0 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> #######################################################################
> # Copyright (c) 2021 Josh Luthy BD Life Sciences - Informatics,
> # Ashland, OR
> #
> # Plugin R Script Template Designed and Written by: Ian Taylor and Josh Luthy
> #
> # CytoNorm algorithm and associated R package developed by Sofie Van Gassen, PhD
> #
> # License
> # The software is distributed under the terms of the
> # Artistic License 2.0
> # http://www.r-project.org/Licenses/Artistic-2.0
> #
> # Disclaimer
> # This software and documentation come with no warranties of any kind.
> # This software is provided "as is" and any express or implied
> # warranties, including, but not limited to, the implied warranties of
> # merchantability and fitness for a particular purpose are disclaimed.
> # In no event shall the  copyright holder be liable for any direct,
> # indirect, incidental, special, exemplary, or consequential damages
> # (including but not limited to, procurement of substitute goods or
> # services; loss of use, data or profits; or business interruption)
> # however caused and on any theory of liability, whether in contract,
> # strict liability, or tort arising in any way out of the use of this
> # software.
> ######################################################################
> 
> options(warn = - 1)
> 
> ## INSTALL REQUIRED PACKAGES ##
> #######################
> # Install from CRAN
> 
> ## TRY TO AUTO INSTALL ##
> 
> # Install function for packages
> packages<-function(x){
+   x<-as.character(match.call()[[2]])
+   if (!require(x,character.only=TRUE)){
+     install.packages(pkgs=x,repos="http://cran.r-project.org")
+     require(x,character.only=TRUE)
+   }
+ }
> 
> # Try to write a special function for devtools installs:
> devPackages<-function(x){
+   if (!require(x,character.only=TRUE)){
+     suppressMessages(suppressWarnings(devtools::install_github(x)))
+   }else{
+     require(x,character.only=TRUE)
+   }
+ }
> 
> fCorePack<-function(x){
+   x<-as.character(match.call()[[2]])
+   if (!require(x,character.only=TRUE)){
+     BiocManager::install(x)
+     require(x,character.only=TRUE)
+   }
+ }
> 
> packages(devtools)
Loading required package: devtools
Loading required package: usethis
> packages(BiocManager)
Loading required package: BiocManager
Bioconductor version '3.13' is out-of-date; the current release version '3.16'
  is available with R version '4.2'; see https://bioconductor.org/install

Attaching package: 'BiocManager'

The following object is masked from 'package:devtools':

    install

> packages(dplyr)
Loading required package: dplyr

Attaching package: 'dplyr'

The following objects are masked from 'package:stats':

    filter, lag

The following objects are masked from 'package:base':

    intersect, setdiff, setequal, union

> fCorePack(FlowSOM)
Loading required package: FlowSOM
Loading required package: igraph

Attaching package: 'igraph'

The following objects are masked from 'package:dplyr':

    as_data_frame, groups, union

The following objects are masked from 'package:stats':

    decompose, spectrum

The following object is masked from 'package:base':

    union

Error: package or namespace load failed for 'FlowSOM' in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Users/zacharylanzar/Library/R/x86_64/4.1/library/CytoML/libs/CytoML.so':
  dlopen(/Users/zacharylanzar/Library/R/x86_64/4.1/library/CytoML/libs/CytoML.so, 0x0006): Library not loaded: /usr/local/opt/libxml2/lib/libxml2.2.dylib
  Referenced from: /Users/zacharylanzar/Library/R/x86_64/4.1/library/CytoML/libs/CytoML.so
  Reason: tried: '/usr/local/opt/libxml2/lib/libxml2.2.dylib' (no such file), '/Library/Frameworks/R.framework/Resources/lib/libxml2.2.dylib' (no such file), '/Library/Java/JavaVirtualMachines/jdk-14.0.2.jdk/Contents/Home/lib/server/libxml2.2.dylib' (no such file)
Bioconductor version 3.13 (BiocManager 1.30.16), R 4.1.1 (2021-08-10)
Installation paths not writeable, unable to update packages
  path: /Library/Frameworks/R.framework/Versions/4.1/Resources/library
  packages:
    BBmisc, BH, BiocManager, DBI, DT, DiagrammeR, FNN, Hmisc, MASS, Matrix,
    MatrixModels, ParamHelpers, R.methodsS3, R.oo, R.utils, RColorBrewer,
    RCurl, RJSONIO, RSQLite, RSpectra, Rcpp, RcppAnnoy, RcppArmadillo,
    RcppEigen, RcppHNSW, Rtsne, Seurat, SeuratObject, XML, ape, aplot, audio,
    babelgene, backports, bit, blob, boot, brew, brio, broom, bslib, callr,
    car, carData, caret, checkmate, class, cli, clipr, cluster, colorspace,
    commonmark, conquer, corrplot, cpp11, crayon, credentials, crosstalk, curl,
    data.table, dbplyr, deldir, desc, devtools, digest, dplyr, dtplyr, e1071,
    evaluate, fansi, farver, fdrtool, fitdistrplus, fontawesome, forcats,
    foreach, foreign, formatR, fs, future, future.apply, gargle, generics,
    gert, ggbeeswarm, ggforce, ggfun, ggplot2, ggpubr, ggraph, ggrepel,
    ggridges, ggsignif, gh, gitcreds, globals, glue, googlesheets4, gower,
    gplots, graphlayouts, gt, gtable, gtools, haven, highr, hms, htmlTable,
    htmltools, htmlwidgets, httpuv, httr, ica, igraph, ipred, irlba, isoband,
    iterators, jpeg, jsonlite, knitr, lattice, latticeExtra, lava, leiden, lhs,
    lifecycle, listenv, lme4, lmtest, locfit, lubridate, magrittr, maptools,
    matrixStats, memoise, mgcv, minqa, mlr, mlrMBO, modelr, msigdbr, nlme,
    nloptr, nnet, openssl, openxlsx, parallelly, patchwork, pbapply, pillar,
    pkgbuild, pkgload, plotly, plyr, png, polyclip, polynom, processx,
    progressr, proxy, ps, purrr, quantreg, randomForest, readr, readxl,
    recipes, remotes, renv, reprex, restfulr, reticulate, rio, rjson, rlang,
    rmarkdown, roxygen2, rpart, rprojroot, rstatix, rstudioapi, rversions,
    rvest, sass, scales, scattermore, scatterpie, sctransform, sessioninfo,
    shadowtext, shiny, sp, spatial, spatstat.data, spatstat.geom,
    spatstat.sparse, spatstat.utils, statmod, stringi, stringr, survival, sys,
    tensorflow, testthat, tfruns, tibble, tidygraph, tidyr, tidyselect,
    tidytree, tidyverse, timeDate, tinytex, tweenr, tzdb, usethis, uuid, uwot,
    vctrs, viridisLite, visNetwork, vroom, waldo, whisker, withr, writexl,
    xfun, xml2, yaml, yulab.utils, zip, zoo
Loading required package: FlowSOM
Error: package or namespace load failed for 'FlowSOM' in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Users/zacharylanzar/Library/R/x86_64/4.1/library/CytoML/libs/CytoML.so':
  dlopen(/Users/zacharylanzar/Library/R/x86_64/4.1/library/CytoML/libs/CytoML.so, 0x0006): Library not loaded: /usr/local/opt/libxml2/lib/libxml2.2.dylib
  Referenced from: /Users/zacharylanzar/Library/R/x86_64/4.1/library/CytoML/libs/CytoML.so
  Reason: tried: '/usr/local/opt/libxml2/lib/libxml2.2.dylib' (no such file), '/Library/Frameworks/R.framework/Resources/lib/libxml2.2.dylib' (no such file), '/Library/Java/JavaVirtualMachines/jdk-14.0.2.jdk/Contents/Home/lib/server/libxml2.2.dylib' (no such file)
> fCorePack(CytoML)
Loading required package: CytoML
Error: package or namespace load failed for 'CytoML' in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Users/zacharylanzar/Library/R/x86_64/4.1/library/CytoML/libs/CytoML.so':
  dlopen(/Users/zacharylanzar/Library/R/x86_64/4.1/library/CytoML/libs/CytoML.so, 0x0006): Library not loaded: /usr/local/opt/libxml2/lib/libxml2.2.dylib
  Referenced from: /Users/zacharylanzar/Library/R/x86_64/4.1/library/CytoML/libs/CytoML.so
  Reason: tried: '/usr/local/opt/libxml2/lib/libxml2.2.dylib' (no such file), '/Library/Frameworks/R.framework/Resources/lib/libxml2.2.dylib' (no such file), '/Library/Java/JavaVirtualMachines/jdk-14.0.2.jdk/Contents/Home/lib/server/libxml2.2.dylib' (no such file)
Bioconductor version 3.13 (BiocManager 1.30.16), R 4.1.1 (2021-08-10)
Installation paths not writeable, unable to update packages
  path: /Library/Frameworks/R.framework/Versions/4.1/Resources/library
  packages:
    BBmisc, BH, BiocManager, DBI, DT, DiagrammeR, FNN, Hmisc, MASS, Matrix,
    MatrixModels, ParamHelpers, R.methodsS3, R.oo, R.utils, RColorBrewer,
    RCurl, RJSONIO, RSQLite, RSpectra, Rcpp, RcppAnnoy, RcppArmadillo,
    RcppEigen, RcppHNSW, Rtsne, Seurat, SeuratObject, XML, ape, aplot, audio,
    babelgene, backports, bit, blob, boot, brew, brio, broom, bslib, callr,
    car, carData, caret, checkmate, class, cli, clipr, cluster, colorspace,
    commonmark, conquer, corrplot, cpp11, crayon, credentials, crosstalk, curl,
    data.table, dbplyr, deldir, desc, devtools, digest, dplyr, dtplyr, e1071,
    evaluate, fansi, farver, fdrtool, fitdistrplus, fontawesome, forcats,
    foreach, foreign, formatR, fs, future, future.apply, gargle, generics,
    gert, ggbeeswarm, ggforce, ggfun, ggplot2, ggpubr, ggraph, ggrepel,
    ggridges, ggsignif, gh, gitcreds, globals, glue, googlesheets4, gower,
    gplots, graphlayouts, gt, gtable, gtools, haven, highr, hms, htmlTable,
    htmltools, htmlwidgets, httpuv, httr, ica, igraph, ipred, irlba, isoband,
    iterators, jpeg, jsonlite, knitr, lattice, latticeExtra, lava, leiden, lhs,
    lifecycle, listenv, lme4, lmtest, locfit, lubridate, magrittr, maptools,
    matrixStats, memoise, mgcv, minqa, mlr, mlrMBO, modelr, msigdbr, nlme,
    nloptr, nnet, openssl, openxlsx, parallelly, patchwork, pbapply, pillar,
    pkgbuild, pkgload, plotly, plyr, png, polyclip, polynom, processx,
    progressr, proxy, ps, purrr, quantreg, randomForest, readr, readxl,
    recipes, remotes, renv, reprex, restfulr, reticulate, rio, rjson, rlang,
    rmarkdown, roxygen2, rpart, rprojroot, rstatix, rstudioapi, rversions,
    rvest, sass, scales, scattermore, scatterpie, sctransform, sessioninfo,
    shadowtext, shiny, sp, spatial, spatstat.data, spatstat.geom,
    spatstat.sparse, spatstat.utils, statmod, stringi, stringr, survival, sys,
    tensorflow, testthat, tfruns, tibble, tidygraph, tidyr, tidyselect,
    tidytree, tidyverse, timeDate, tinytex, tweenr, tzdb, usethis, uuid, uwot,
    vctrs, viridisLite, visNetwork, vroom, waldo, whisker, withr, writexl,
    xfun, xml2, yaml, yulab.utils, zip, zoo
Loading required package: CytoML
Error: package or namespace load failed for 'CytoML' in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Users/zacharylanzar/Library/R/x86_64/4.1/library/CytoML/libs/CytoML.so':
  dlopen(/Users/zacharylanzar/Library/R/x86_64/4.1/library/CytoML/libs/CytoML.so, 0x0006): Library not loaded: /usr/local/opt/libxml2/lib/libxml2.2.dylib
  Referenced from: /Users/zacharylanzar/Library/R/x86_64/4.1/library/CytoML/libs/CytoML.so
  Reason: tried: '/usr/local/opt/libxml2/lib/libxml2.2.dylib' (no such file), '/Library/Frameworks/R.framework/Resources/lib/libxml2.2.dylib' (no such file), '/Library/Java/JavaVirtualMachines/jdk-14.0.2.jdk/Contents/Home/lib/server/libxml2.2.dylib' (no such file)
> # devPackages("saeyslab/FlowSOM")
> devPackages("saeyslab/CytoNorm")
Loading required package: saeyslab/CytoNorm
> library("CytoNorm")
> # fCorePack(FlowSOM)
> packages(ggplot2)
Loading required package: ggplot2
> packages(pheatmap)
Loading required package: pheatmap
> #######################
> ###############################
> ##         LOAD INPUTS       ##
> ###############################
> 
> isWindows <- FALSE
> outPath <- "/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/CytoNorm_Dataset_1674154397010/_Users_zacharylanzar_Documents_CHunter_lab_ZL_23_Treg_rebound_Kinetics_#2_Data_TcellPhenotyping_CompensationCorrected_Spleen_11323_forBatchCorrection.CytoNorm.csv"
> 
> if (isWindows){
+   outPath <- gsub("/", "\\\\", outPath)
+ }
> 
> tempDir <- paste0(outPath,"/temp")
> 
> if (isWindows){
+   tempDir <- gsub("/", "\\\\", tempDir)
+ }
> 
> inCSVPath <- "/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection"
> sampleName <- "sampleName"
> applyOn <- "NONE"
> 
> # TODO: Settings being imported from Java - switches can be repurposed for needs in CytoNorm
> timeStamp <- "1674154397010"
> 
> ## Repurposing 'FJ FLIP GRAPH' to the Hashmap of 'samples : control' status
> hashList <- list("/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/concatSCC_D10_1.fcs:true","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/concatSCC_D22_1.fcs:true","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/concatSCC_D35_1.fcs:true","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/concatSCC_D60_1.fcs:true","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day10_Spleen_d10_Infected_1.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day10_Spleen_d10_Infected_2.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day10_Spleen_d10_Infected_3.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day10_Spleen_d10_Infected_4.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day10_Spleen_d10_Infected_5.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day22_Spleen_d22_Infected_001.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day22_Spleen_d22_Infected_002.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day22_Spleen_d22_Infected_003.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day22_Spleen_d22_Infected_004.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day22_Spleen_d22_Infected_005.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day35_Spleen_Infected_001.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day35_Spleen_Infected_002.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day35_Spleen_Infected_003.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day35_Spleen_Infected_004.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day60_Spleen_Infected_001.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day60_Spleen_Infected_002.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/D+ ata/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day60_Spleen_Infected_003.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day60_Spleen_Infected_004.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day60_Spleen_Infected_005.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day10_Spleen_d10_PBS_1.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day10_Spleen_d10_PBS_2.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day10_Spleen_d10_PBS_3.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day10_Spleen_d10_PBS_4.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day10_Spleen_d10_PBS_5.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day22_Spleen_d22_Naive_001.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day22_Spleen_d22_Naive_002.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day22_Spleen_d22_Naive_003.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day22_Spleen_d22_Naive_004.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day22_Spleen_d22_Naive_005.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day35_Spleen_Naive_001.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day35_Spleen_Naive_002.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day35_Spleen_Naive_003.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day35_Spleen_Naive_004.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day35_Spleen_Naive_005.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day60_Spleen_Naive_001.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day60_Spleen_Naive_002.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day60_Spl+ een_Naive_003.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day60_Spleen_Naive_004.fcs:false","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day60_Spleen_Naive_005.fcs:false")
> 
> nCluster <- 5
> 
> ## Repurposing 'FJ TRIM' to ... batches
> batchList <- list("/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/concatSCC_D10_1.fcs:01","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/concatSCC_D22_1.fcs:02","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/concatSCC_D35_1.fcs:03","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/concatSCC_D60_1.fcs:04","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day10_Spleen_d10_Infected_1.fcs:01","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day10_Spleen_d10_Infected_2.fcs:01","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day10_Spleen_d10_Infected_3.fcs:01","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day10_Spleen_d10_Infected_4.fcs:01","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day10_Spleen_d10_Infected_5.fcs:01","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day22_Spleen_d22_Infected_001.fcs:02","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day22_Spleen_d22_Infected_002.fcs:02","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day22_Spleen_d22_Infected_003.fcs:02","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day22_Spleen_d22_Infected_004.fcs:02","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day22_Spleen_d22_Infected_005.fcs:02","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day35_Spleen_Infected_001.fcs:03","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day35_Spleen_Infected_002.fcs:03","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day35_Spleen_Infected_003.fcs:03","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day35_Spleen_Infected_004.fcs:03","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day60_Spleen_Infected_001.fcs:04","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day60_Spleen_Infected_002.fcs:04","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323+ /forBatchCorrection/export_Infected_Day60_Spleen_Infected_003.fcs:04","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day60_Spleen_Infected_004.fcs:04","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Infected_Day60_Spleen_Infected_005.fcs:04","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day10_Spleen_d10_PBS_1.fcs:01","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day10_Spleen_d10_PBS_2.fcs:01","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day10_Spleen_d10_PBS_3.fcs:01","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day10_Spleen_d10_PBS_4.fcs:01","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day10_Spleen_d10_PBS_5.fcs:01","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day22_Spleen_d22_Naive_001.fcs:02","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day22_Spleen_d22_Naive_002.fcs:02","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day22_Spleen_d22_Naive_003.fcs:02","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day22_Spleen_d22_Naive_004.fcs:02","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day22_Spleen_d22_Naive_005.fcs:02","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day35_Spleen_Naive_001.fcs:03","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day35_Spleen_Naive_002.fcs:03","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day35_Spleen_Naive_003.fcs:03","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day35_Spleen_Naive_004.fcs:03","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day35_Spleen_Naive_005.fcs:03","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day60_Spleen_Naive_001.fcs:04","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day60_Spleen_Naive_002.fcs:04","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day60_Spleen_Naive_003.fcs:04","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyp+ ing/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day60_Spleen_Naive_004.fcs:04","/Users/zacharylanzar/Documents/CHunter_lab/ZL_23_Treg_rebound_Kinetics_#2/Data/TcellPhenotyping/CompensationCorrected_Spleen_11323/forBatchCorrection/export_Naive_Day60_Spleen_Naive_005.fcs:04")
> parLegendPosition <- "FJ_LEGEND_POS" ## "left", "top", "right", "bottom"
> catParRaw <- "FJ_CAT_PAR"
> # createStatTable <- FJ_STATS
> 
> ## This should be the number of samples (or batches) we're trying to correct
> numSamples <- length(hashList)
> 
> ## Get the DataSet Ready for CytoNorm ##
> ########################################
> 
> outPuts <- paste0(inCSVPath, "/CytoNorm_Dataset_", timeStamp)
> 
> # Let's get the FCS file lists addressed the way CytoNorm expects to find them.
> 
> data02 <- data.frame( File = character(),
+                       Path = character(),
+                       Type = character(),
+                       Batch = character(),
+                       stringsAsFactors = F )
> 
> for (i in 1:numSamples) {
+   ithHash <- as.character(hashList[i])
+ 
+   ithBatchHash <- as.character(batchList[i])
+ 
+   # ithHash <- as.character(hashList[1])
+   # tidyselect::vars_select(ithHash, contains(":true"))
+   # ":true" %in% ithHash
+   # grepl(":true", ithHash)
+   # sub(".*/", "", sub(":true","",ithHash))
+ 
+   if (grepl(":true", ithHash)) {   # }, ignore.case = F, vars = ithHash)) {
+     # put these samples into the control group (type = 1)
+     data02[i,] <- data.frame( File = sub(".*/", "", sub(":true","",ithHash)),
+                               Path = sub(":true","",ithHash),
+                               Type = "1",
+                               Batch = sub(".*:", "", ithBatchHash),
+                               stringsAsFactors = F )
+ 
+ 
+   } else {
+     # put these samples inot the test group (type = 2)
+     data02[i,] <- data.frame( File = sub(".*/", "", sub(":false","",ithHash)),
+                               Path = sub(":false","",ithHash),
+                               Type = "2",
+                               Batch = sub(".*:", "", ithBatchHash),
+                               stringsAsFactors = F )
+   }
+ }
> 
> #   ithControl <- gsub(":*", "", ithHash)
> #   split(hashList[1], ':', drop = F)
> #   splitR <- split(hashList)
> 
> data <- data02
> 
> ########################################
> ########################################
> 
> 
> 
> ## Initial Treatment of Dataset ##
> ##################################
> 
> data$Type <- c("1" = "Train", "2" = "Validation")[data$Type]
> 
> train_data <- dplyr::filter(data, Type == "Train")
> validation_data <- dplyr::filter(data, Type == "Validation")
> 
> ff <- flowCore::read.FCS(data$Path[1])
> 
> channels <- flowCore::colnames(ff)
> transformList <- flowCore::transformList(channels,
+                                          CytoNorm::cytofTransform)
> 
> transformList.reverse <- flowCore::transformList(channels,
+                                                  CytoNorm::cytofTransform.reverse)
> 
> 
> ##  Step 02 of Initial Workflow ##
> ##################################
> 
> fsom <- CytoNorm::prepareFlowSOM(train_data$Path,
+                                  channels,
+                                  nCells = 6000,
+                                  FlowSOM.params = list(xdim = 5,
+                                                        ydim = 5,
+                                                        nClus = nCluster,
+                                                        scale = FALSE), #TODO place 'colsToUse' here for user selected params
+                                  transformList = transformList,
+                                  seed = 1)
Error in dyn.load(file, DLLpath = DLLpath, ...) : 
  unable to load shared object '/Users/zacharylanzar/Library/R/x86_64/4.1/library/CytoML/libs/CytoML.so':
  dlopen(/Users/zacharylanzar/Library/R/x86_64/4.1/library/CytoML/libs/CytoML.so, 0x0006): Library not loaded: /usr/local/opt/libxml2/lib/libxml2.2.dylib
  Referenced from: /Users/zacharylanzar/Library/R/x86_64/4.1/library/CytoML/libs/CytoML.so
  Reason: tried: '/usr/local/opt/libxml2/lib/libxml2.2.dylib' (no such file), '/Library/Frameworks/R.framework/Resources/lib/libxml2.2.dylib' (no such file), '/Library/Java/JavaVirtualMachines/jdk-14.0.2.jdk/Contents/Home/lib/server/libxml2.2.dylib' (no such file)
Calls: <Anonymous> ... asNamespace -> loadNamespace -> library.dynam -> dyn.load
Execution halted

The flowjo plug in is able to get as far as the beginning of step 2, but early on it looks to me as though FlowSOM and CytoML are not loading properly? any help would be greatly appreciated.

Thank you!

@SofieVG
Copy link
Member

SofieVG commented Jan 20, 2023 via email

@BaptLabRight
Copy link
Author

Hi Sofie,
Thank you for your reply.
In my hands, library("CytoML") works perfectly.

I had to correct the R script produced by Flowjo by myself to make it able to produce normalized fcs files:
In fact in Step 02 "Initial WorkFlow", I needed to had "emptyValue=F"

See code below

Step 02 of Initial Workflow

##################################
fsom <- CytoNorm::prepareFlowSOM(train_data$Path,emptyValue=F,
channels,
nCells = 6000,
FlowSOM.params = list(xdim = 5,
ydim = 5,
nClus = nCluster,
scale = FALSE), #TODO place 'colsToUse' here for user selected params
transformList = transformList,
seed = 1)

cVals = sort(c(5,nCluster,10,15))
png("NUL")
pdf("NUL")

cvs <- testCV(fsom, cluster_values = cVals, plot=FALSE)

p <- PlotOverviewCV(fsom, cvs)

ggsave(paste0(outPuts,"/CVcompare.png"), p)

dev.off()

eval(parse(text=paste0("cvs","$","pctgs","$","",nCluster,"")))

In addition, I needed to add emptyValue=F in Step 03. See code below

model <- CytoNorm.train(files = train_data$Path,emptyValue=F,
labels = train_data$Batch,
channels = channels,
transformList = transformList,
outputDir = tempDir,
# plot = FJ_SOMMARY_PLOTS, TODO: implement this option as checkbox (summary plots).
FlowSOM.params = list(nCells = 10000,
xdim = 10,
ydim = 10,
nClus = nCluster,
scale = FALSE), #TODO use colsToUse to train on selected params
normMethod.train = QuantileNorm.train,
normParams = list(nQ = 101,
goal = "mean"),
seed = 1,
verbose = TRUE)

Same thing for Step 04

CytoNorm.normalize(model = model,
files = validation_data$Path,
labels = validation_data$Batch,
emptyValue=F,

                transformList = transformList,
                transformList.reverse = transformList.reverse,
                normMethod.normalize = QuantileNorm.normalize,
                outputDir = OutPutDir,
                prefix = "Norm_",
                clean = TRUE,
                verbose = TRUE)

With these additions, the script produces normalized fcs files.

Could you confirm these corrections?

Thank you

@SofieVG
Copy link
Member

SofieVG commented Jan 20, 2023 via email

@zlanzar
Copy link

zlanzar commented Jan 23, 2023

Hi Sophie,

I removed my previous installation of CytoML and re-installed using the commands you listed above:

if (!require("BiocManager", quietly = TRUE)) install.packages("BiocManager")
BiocManager::install("CytoML")

Version 2.4.0 of CytoML is installed and is loadable with library("CytoML") in R, but the R.output script is still throwing the same error.

@SofieVG
Copy link
Member

SofieVG commented Jan 23, 2023 via email

@zlanzar
Copy link

zlanzar commented Jan 23, 2023 

> library(CytoML)
> R.home()
[1] "/Library/Frameworks/R.framework/Resources"

Screen Shot 2023-01-23 at 12 43 22 PM

When using that path to R it throws the unable to find CytoML error.

I used the .libPaths() to find where my R library is

> .libPaths()
[1] "/Users/zacharylanzar/Library/R/x86_64/4.1/library"             
[2] "/Library/Frameworks/R.framework/Versions/4.1/Resources/library"

Pathing to either above allows the program to run. No errors are thrown but there is no R.output nor files generated.

@Heather-WHH Heather-WHH mentioned this issue Apr 29, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@SofieVG @zlanzar @BaptLabRight