"SYSargsList"
SYSargsList-class.Rd
SYSargsList
S4 class is a list-like container where each instance stores
all the input/output paths and parameter components required for a particular
data analysis step based on command-line- or R-based software.
SYSargsList
instances are constructed by the SYSargsList
function.
## Accessors
stepsWF(x)
statusWF(x)
targetsWF(x)
outfiles(x)
SE(x, ...)
dependency(x)
projectInfo(x)
runInfo(x)
## Methods
cmdlist(x, ...)
codeLine(x, ...)
SampleName(x, ...)
stepName(x)
baseCommand(x, ...)
targetsheader(x, ...)
yamlinput(x, ...)
viewEnvir(x, silent = FALSE)
copyEnvir(x, list = character(), new.env = globalenv(), silent = FALSE)
addResources(x, step, resources)
## Subset Methods
subset(x, ...)
getColumn(x, step, position = c("outfiles", "targetsWF"), column = 1,
names = SampleName(x, step))
## Replacement
appendStep(x, after = length(x), ...) <- value
yamlinput(x, paramName, ...) <- value
replaceStep(x, step, step_name = "default") <- value
renameStep(x, step, ...) <- value
dependency(x, step, ...) <- value
appendCodeLine(x, after = length(x), ...) <- value
replaceCodeLine(x, line, ...) <- value
updateColumn(x, step, position = c("outfiles", "targetsWF")) <- value
An instance of class SYSargsList
.
character or numeric. Workflow step name or position index.
If set to TRUE
, all messages returned by the function will be suppressed.
a character vector naming objects to be copyied from the enviroment.
An environment to copy to. Default is globalenv()
.
List
for reserving for each cluster job sufficient computing resources
including memory (Megabyte), number of nodes (Int), CPU cores, walltime (Minutes),
etc. It is necessary two additional files: conffile
and template
.
conffile
is the path to configuration file (default location
./.batchtools.conf.R
). This file contains in its simplest form just one
command, such as this line for the scheduler:
cluster.functions <- makeClusterFunctionsSlurm(template="batchtools.slurm.tmpl")
.
For more detailed information visit this page: https://mllg.github.io/batchtools/index.html.
template
The template files for a specific queueing/scheduling systems
can be downloaded from here:
https://github.com/mllg/batchtools/tree/master/inst/templates. Slurm,
PBS/Torque, and Sun Grid Engine (SGE) templates are provided within the package
demo data.
character. Options are "outfiles" or "targetsWF"
slots.
character or numeric. Which column will be subset from the position
argument.
character vector. Names of the workflow step.
A subscript, after which the values are to be appended.
character. Input name from yamlinput(x)
, which value should be replaced.
character with the new step name. Default value will automatically give a name:
step_<step index position>
.
numeric. Index position of the code line to be added or replaced.
object containing the values to be replaced to SYSargsList
. Values may
be of the same class as the orginal values. For updateColumn<-
a
DataFrame
must have the same rows as the original DataFrame
.
However, if there is no column/rows in the original DataFrame
, the new
DataFrame
will replace the empty one. If there is a non-empty
DataFrame
, any existing columns with the same name as this new
DataFrame
will be replaced. Any columns that do not exist in the
original DataFrame
will be added to the original DataFrame
.
Further arguments to be passed to or from other methods.
Objects can be created by calls of the form new("SYSargsList", ...)
.
stepsWF
:Object of class "list"
storing all the
steps objects of the workflow. Each step can either be SYSargs2
or
LineWise
.
statusWF
:Object of class "list"
storing all the
success and failure of each step in the workflow.
targetsWF
:Object of class "list"
storing all the
targets DataFrame
for each step in the workflow. For the
LineWise
steps, a DataFrame
with 0 rows and 0 columns will
be displayed.
outfiles
:Object of class "list"
storing all the
output DataFrame
for each step in the workflow. For the
LineWise
steps, a DataFrame
with 0 rows and 0 columns will be
displayed.
SE
:Object of class "list"
storing all the
SummarizedExperiment
objects in the workflow.
dependency
:Object of class "list"
storing all the
dependency graphs in the workflow.
projectInfo
:Object of class "list"
storing all the
projectInfo
information of the workflow.
runInfo
:Object of class "list"
storing all the
runInfo
information of each step in the workflow.
targets_connection
:Object of class "list"
storing all
targets files connection in the workflow.
signature(x = "SYSargsList", i = "ANY", j = "ANY", drop = "ANY")
:
subsetting of class with bracket operator
signature(x = "SYSargsList", i = "ANY", j = "ANY")
:
subsetting of class with bracket operator
signature(x = "SYSargsList")
:
extracting slots elements by name
signature(from = "list", to = "SYSargsList")
:
as(list, "SYSargsList")
signature(from = "SYSargsList", to = "list")
:
as(SYSargsList, "list")
signature(x = "SYSargsList")
: Coerce back to
list as(SYSargsList, "list")
signature(x = "SYSargsList")
: extracts number of
SYSargsList
steps
signature(x = "SYSargsList")
: extracts slot names
signature(object = "SYSargsList")
: summary view of
SYSargsList
steps
signature(x = "SYSargsList")
: extract data from
stepsWF
slot
signature(x = "SYSargsList")
: extract data from
statusWF
slot
signature(x = "SYSargsList")
: extract data from
targetsWF
slot
signature(x = "SYSargsList")
: extract data from
outfiles
slot
signature(x = "SYSargsList")
: extract data from
SE
slot
signature(x = "SYSargsList")
: extract data from
dependency
slot
signature(x = "SYSargsList")
: extract data from
projectInfo
slot
signature(x = "SYSargsList")
: extract data from
runInfo
slot
signature(x = "SYSargsList", ...)
:
extracts data from cmdlist
slot for each SYSargs2
step
signature(x = "SYSargsList", step)
:
extracts data from codeLine
slot for LineWise
step
signature(x = "SYSargsList", step)
:
extracts Sample ID from SYSargs2
instance step
signature(x = "SYSargsList")
:
extracts steps names from workflow instance
signature(x = "SYSargsList", step)
:
extracts baseCommand
from SYSargs2
instance step
signature(x = "SYSargsList", step)
:
extracts targetsheader
from SYSargs2
instance step
signature(x = "SYSargsList", step)
:
extracts data from yamlinput
slot for each SYSargs2
step
signature(x = "SYSargsList", silent = FALSE)
:
return a vector of character strings giving the names of the
objects in the SYSargsList
environment
signature(x = "SYSargsList", list = character(),
new.env = globalenv(), silent = FALSE)
:
copy of the contents or select objects from SYSargsList
environment and place them into new.env
signature(x = "SYSargsList", step, resources)
:
Adds the computing resources for one or multiple steps in the workflow.
If the particular step(s) is set to be executed "management section,"
when the resources is added, the step(s) will be executed on the
"compute section."
signature(x = "SYSargsList", step,
position = c("outfiles", "targetsWF"), column = 1,
names = SampleName(x, step))
:
extracts the information for targetsWF
or outfiles
slots. The information can be used in an R code downstream
signature(x = "SYSargsList", i = "ANY", j = "ANY", value = "ANY")
:
replacement method for SYSargsList
class
signature(x = "SYSargsList", after = length(x))
:
insert the SYSargsList
or LineWise
object onto x at
the position given by after
signature(x = "SYSargsList", step, paramName )
:
replace a value in the yamlinput
slot for a specific step instance
signature(x = "SYSargsList", step, step_name = "default")
:
replace a specific step in the workflow instance
signature(x = "SYSargsList")
:
rename a stepName
in the workflow instance
signature(x = "SYSargsList", step)
:
replace dependency graph for a specific step instance
signature(x = "SYSargsList", step, after = length(x))
:
insert the R code in a specific step at the position given by after
signature(x = "SYSargsList", step, line)
:
replace the R code in a specific step at the position given by line
signature(x = "SYSargsList", step,
position = c("outfiles", "targetsWF"))
:
update or add a new column in targetsWF
or outfiles
slots
SYSargs2
, LineWise
, and SPRproject
sal <- SPRproject(overwrite=TRUE)
#> Recreating directory '/home/runner/work/systemPipeR/systemPipeR/docs/reference/.SPRproject'
#> Creating file '/home/runner/work/systemPipeR/systemPipeR/docs/reference/.SPRproject/SYSargsList.yml'
targetspath <- system.file("extdata/cwl/example/targets_example.txt",
package="systemPipeR")
## Constructor and `appendStep<-`
appendStep(sal) <- SYSargsList(step_name = "echo",
targets=targetspath, dir=TRUE,
wf_file="example/workflow_example.cwl",
input_file="example/example.yml",
dir_path = system.file("extdata/cwl",
package="systemPipeR"),
inputvars = c(Message = "_STRING_",
SampleName = "_SAMPLE_"))
appendStep(sal) <- LineWise(code = {
hello <- lapply(getColumn(sal, step=1, 'outfiles'), function(x) yaml::read_yaml(x))
},
step_name = "R_read",
dependency = "echo")
sal
#> Instance of 'SYSargsList':
#> WF Steps:
#> 1. echo --> Status: Pending
#> Total Files: 3 | Existing: 0 | Missing: 3
#> 1.1. echo
#> cmdlist: 3 | Pending: 3
#> 2. R_read --> Status: Pending
#>
length(sal)
#> [1] 2
names(sal)
#> [1] "stepsWF" "statusWF" "targetsWF"
#> [4] "outfiles" "SE" "dependency"
#> [7] "targets_connection" "projectInfo" "runInfo"
## Accessors
stepsWF(sal)
#> $echo
#> Instance of 'SYSargs2':
#> Slot names/accessors:
#> targets: 3 (M1...M3), targetsheader: 1 (lines)
#> modules: 0
#> wf: 1, clt: 1, yamlinput: 3 (inputs)
#> input: 3, output: 3
#> cmdlist: 3
#> Sub Steps:
#> 1. echo (rendered: TRUE)
#>
#>
#>
#> $R_read
#> Instance of 'LineWise'
#> Code Chunk length: 1
#>
statusWF(sal)
#> $echo
#> DataFrame with 3 rows and 5 columns
#> Targets Total_Files Existing_Files Missing_Files echo
#> <character> <numeric> <numeric> <numeric> <matrix>
#> M1 M1 1 0 1 Pending
#> M2 M2 1 0 1 Pending
#> M3 M3 1 0 1 Pending
#>
#> $R_read
#> DataFrame with 1 row and 2 columns
#> Step status.summary
#> <character> <character>
#> 1 R_read Pending
#>
targetsWF(sal)
#> $echo
#> DataFrame with 3 rows and 2 columns
#> Message SampleName
#> <character> <character>
#> M1 Hello World! M1
#> M2 Hello USA! M2
#> M3 Hello Bioconductor! M3
#>
#> $R_read
#> DataFrame with 0 rows and 0 columns
#>
outfiles(sal)
#> $echo
#> DataFrame with 3 rows and 1 column
#> string
#> <character>
#> M1 results/M1.txt
#> M2 results/M2.txt
#> M3 results/M3.txt
#>
#> $R_read
#> DataFrame with 0 rows and 0 columns
#>
SE(sal)
#> $echo
#> class: SummarizedExperiment
#> dim: 0 3
#> metadata(2): SPRversion targetsheader
#> assays(0):
#> rownames: NULL
#> rowData names(0):
#> colnames(3): M1 M2 M3
#> colData names(2): Message SampleName
#>
#> $R_read
#> NULL
#>
dependency(sal)
#> $echo
#> [1] NA
#>
#> $R_read
#> [1] "echo"
#>
projectInfo(sal)
#> $project
#> [1] "/home/runner/work/systemPipeR/systemPipeR/docs/reference"
#>
#> $data
#> [1] "data"
#>
#> $param
#> [1] "param"
#>
#> $results
#> [1] "results"
#>
#> $logsDir
#> [1] ".SPRproject"
#>
#> $sysargslist
#> [1] ".SPRproject/SYSargsList.yml"
#>
runInfo(sal)
#> $env
#> <environment: 0x55f488d73ef8>
#>
#> $runOption
#> $runOption$echo
#> $runOption$echo$directory
#> [1] TRUE
#>
#> $runOption$echo$run_step
#> [1] "mandatory"
#>
#> $runOption$echo$run_session
#> [1] "management"
#>
#>
#> $runOption$R_read
#> $runOption$R_read$directory
#> [1] FALSE
#>
#> $runOption$R_read$run_step
#> [1] "mandatory"
#>
#> $runOption$R_read$run_session
#> [1] "management"
#>
#> $runOption$R_read$run_remote_resources
#> NULL
#>
#>
#>
## Methods
cmdlist(sal, step=1, targets=1:2) ## SYSargs2 step
#> $echo
#> $echo$M1
#> $echo$M1$echo
#> [1] "echo Hello World! > results/M1.txt"
#>
#>
#> $echo$M2
#> $echo$M2$echo
#> [1] "echo Hello USA! > results/M2.txt"
#>
#>
#>
codeLine(sal, step=2) ## LineWise step
#> R_read
#> hello <- lapply(getColumn(sal, step = 1, "outfiles"), function(x) yaml::read_yaml(x))
SampleName(sal, step="echo")
#> [1] "M1" "M2" "M3"
stepName(sal)
#> [1] "echo" "R_read"
baseCommand(sal, 1) ## SYSargs2 step
#> $echo
#> [1] "echo"
#>
targetsheader(sal, step=1) ## SYSargs2 step
#> $targetsheader
#> [1] "# Project ID: SPR Example"
#>
yamlinput(sal, step=1) ## SYSargs2 step
#> $message
#> [1] "_STRING_"
#>
#> $SampleName
#> [1] "_SAMPLE_"
#>
#> $results_path
#> $results_path$class
#> [1] "Directory"
#>
#> $results_path$path
#> [1] "./results"
#>
#>
viewEnvir(sal)
#> <environment: 0x55f488d73ef8>
#> character(0)
copyEnvir(sal, list = character(), new.env = globalenv())
#> <environment: 0x55f488d73ef8>
#> Copying to 'new.env':
resources <- list(conffile= system.file("extdata/.batchtools.conf.R",
package="systemPipeR"),
template= system.file("extdata/batchtools.slurm.tmpl",
package="systemPipeR"),
Njobs=3, ## Usually, the samples number
walltime=60, ## minutes
ntasks=1,
ncpus=4,
memory=1024 ## Mb
)
addResources(sal, 1, resources= resources)
#> Please note that the 'echo' step option 'management' was replaced with 'compute'.
#> Instance of 'SYSargsList':
#> WF Steps:
#> 1. echo --> Status: Pending
#> Total Files: 3 | Existing: 0 | Missing: 3
#> 1.1. echo
#> cmdlist: 3 | Pending: 3
#> 2. R_read --> Status: Pending
#>
## Subset Methods
sal_sub <- subset(sal, subset_steps=1, input_targets=1:2, keep_steps = TRUE)
sal_sub
#> Instance of 'SYSargsList':
#> WF Steps:
#> 1. echo --> Status: Pending
#> Total Files: 2 | Existing: 0 | Missing: 2
#> 1.1. echo
#> cmdlist: 2 | Pending: 2
#> 2. R_read --> Status: Pending
#>
targetsIn <- getColumn(sal, step=1, position = c("outfiles"))
targetsIn
#> M1 M2 M3
#> "results/M1.txt" "results/M2.txt" "results/M3.txt"
## Replacement
renameStep(sal, step=1) <- "new_echo"
dependency(sal, step=2) <- "new_echo"
updateColumn(sal, step=2, position = c("targetsWF")) <- data.frame(targetsIn)
targetsWF(sal)
#> $new_echo
#> DataFrame with 3 rows and 2 columns
#> Message SampleName
#> <character> <character>
#> M1 Hello World! M1
#> M2 Hello USA! M2
#> M3 Hello Bioconductor! M3
#>
#> $R_read
#> DataFrame with 3 rows and 1 column
#> targetsIn
#> <character>
#> M1 results/M1.txt
#> M2 results/M2.txt
#> M3 results/M3.txt
#>
replaceStep(sal, step=2) <- LineWise(code = {
hello <- "Printing a new message"
},
step_name = "R_hello",
dependency = "new_echo")
codeLine(sal)
#> new_echo step have been dropped because it is not a LineWise object.
#> R_hello
#> hello <- "Printing a new message"
yamlinput(sal, step=1, paramName="results_path") <- list(results_path=list(
class="Directory", path="./data"))
cmdlist(sal, step = 1, targets = 1)
#> $new_echo
#> $new_echo$M1
#> $new_echo$M1$echo
#> [1] "echo Hello World! > data/M1.txt"
#>
#>
#>
appendCodeLine(sal, step=2, after = 0) <- "log <- log(10)"
codeLine(sal, 2)
#> R_hello
#> log <- log(10)
#> hello <- "Printing a new message"
replaceCodeLine(sal, step=2, line=1) <- LineWise(code = {
log <- log(50)
})
codeLine(sal, 2)
#> R_hello
#> log <- log(50)
#> hello <- "Printing a new message"