print version

file of commands

• file: The file of commands. Any # of lines
  

Specify the input delimiter

• delimiter: Use 'space' for space, 'tab' for tab,'?' to guess between tab and space
  

Input comment

• comment:
  

Use tabs. A shortcut for -delimiter tab

Columns are fixed widths

• widths: w1,w2,...,wN
  

For example, with the below data that has Xs to represent spaces

nameXXXvalue1XXXXXvalue2XXXXX   
foo    1          2
bar    3          4

You would set the -widths to be: seesv -widths 7,11,11

Don't treat quotes as special characters
Normally, when parsing a CSV file if a quote is encountered it is assumed to be a special character enclosing commas or new lines. If the quotes in your CSV file aren't special then use this command.

Input is one text line per row. i.e., no new lines in a data row. Setting this can improve performance on large files

Start at pattern in source file

• start pattern:
  

End at pattern in source file

• stop pattern:
  

Input has a leading byte order mark (BOM) that should be stripped out

Specify the file encoding

• encoding: File Encoding see https://docs.oracle.com/javase/8/docs/technotes/guides/intl/encoding.doc.html values:UTF-8,UTF-16,UTF-16BE,UTF-16LE,UTF-32,UTF-32BE,UTF-32LE,CESU-8,IBM00858,IBM437,IBM775,IBM850,IBM852,IBM855,IBM857,IBM862,IBM866,ISO-8859-1,ISO-8859-13,ISO-8859-15,ISO-8859-2,ISO-8859-4,ISO-8859-5,ISO-8859-7,ISO-8859-9,KOI8-R,KOI8-U,Not available,US-ASCII,windows-1250,windows-1251,windows-1252,windows-1253,windows-1254,windows-1257,x-IBM737,x-IBM874,x-UTF-16LE-BOM,x-UTF-32BE-BOM,x-UTF-32LE-BOM
  

Raw header

• header: Column names
  

Use this if your file does not have a header.

Treat input files separately

• template: File template - ${file_shortname} ${file_name} ${count}
  

Parse the input as json

• arrayPath: Path to the array e.g., obj1.arr[2].obj2
  
• objectPaths: One or more paths to the objects e.g. geometry,features
  

The -json command takes as first argument a path to the array of objects that constitute the data. For example, if you had JSON like:

{
    "p1": {
	"p2": {
	    "values": [
		1,2,3,4
	    ]
	}
    }
}

The command would be: seesv -json p1.p2.values "" Since the values array just holds numeric values there is no column name. The data would be:

1

2

3

4

If you had a path that had an array component that you wanted to access, for example, the second element of the array, e.g.:

{
    "p1": [
	{"foo":"bar"},
	{
	    "p2": {
		"values": [
		    "1","2","3","4"
		]
	    }
	}
    ]
}

The command would be: seesv -json "p1[1\].p2.values" "" Here is an example where the value array holds objects.

{
    "p1": {
	"p2": {
	    "values": [
		{
		    "name":"jim",
		    "value":5
		},
		{
		    "name":"jill",
		    "value":10
		}		
	    ]
	}
    }
}

The command is: seesv -json "p1.p2.values" "" With the result using the object attribute names as column names:

name	value
jim	5
jill	10

If the object within the array is a nested object and you only want to access an inner object, e.g.:

{
    "p1": {
	"p2": {
	    "values": [
		{
		    "foo":"bar",
		    "goodvalues": {
			"name":"jim",
			"value":5
		    }
		},
		{
		    "foo":"bar",
		    "goodvalues": {
			"name":"jill",
			"value":10
		    }
		},
	    ]
	}
    }
}

You would use the second argument to specify an inner path to the object of interest: seesv -json "p1.p2.values" "goodvalues" If the object within the array is a nested object and you only wanted to access multiple inner objects, e.g.:

{
    "p1": {
	"p2": {
	    "values": [
		{
		    "foo":"bar",
		    "goodvalues": {
			"name":"jim",
			"value":5
		    },
		    "moregoodvalues": {
			"age":74,
		    }    
		},
		{
		    "foo":"bar",
		    "goodvalues": {
			"name":"jill",
			"value":10
		    },
		    "moregoodvalues": {
			"age":80,
		    }
		},
	    ]
	}
    }
}

Then the second argument can be a comma separated list of path specifications:

seesv  -json "p1.p2.values" "goodvalues,moregoodvalues"

Resulting in:

age	name	value
74	jim	5
80	jill	10

Join different arrays in the input JSON

• arrayPaths: comma separated list of the array paths
  
• keys: Comma separated list of keys to match on
  
• pattern: Optional pattern to replace the key value with
  
• replace: Pattern replace
  
• missing: Missing value
  

Extract a value from a JSON column

• columns: Column names
  
• arrayPath: Path to the array e.g., obj1.arr[2].obj2
  

Parse the input as geojson

• includePolygon: Include polygon values:true,false
  

Read input from a PDF file.
The RAMADDA_TABULA environment variable needs to be set to the tabula.sh file found in this release

export RAMADDA_TABULA=/path/to/tabula.sh

Parse the input as xml

• path: Path to the elements
  

Parse the input shapefile

• props: "addPoints true addShapes false"
  

The -shapefile command can either take a .shp shapefile or a .zip shapefile. If it is a .zip shapefile that contains a .dbf file then then values within the .dbf file are extracted and added as output. The argument is a set of name/value pairs. For now these can contain an addPoints and an addShapes values, e.g. the below will add a point which is the calculated centroid of the shape. seesv -shapefile "addPoints true" -p test.zip With addShapes=true the polygon will be extracted and added as a column value: seesv -shapefile "addShapes true" -p test.zip

Parse the input as text lines. Treat each line as one column
For each line in the input file this creates a row that contains one column which holds the line.

Parse tables in the input html file

• skip: Number of tables to skip
  
• pattern: Pattern to skip to
  
• properties: Other name value args -
  • numTables N:Number of tables to process. Default is 1
  • removeEntity true:remove HTML entities
  • removePattern pattern
  • extractUrls true
  • columnN.extractUrls true: N=column number
  • stripTags false: strip any HTML tags. Default =true
  • columnN.stripTags false: N=column number. Set stripTags for the column
  
  

If "pattern" is specified then skip to the first line that matches the pattern. Then skip over the "skip" number of tables. Then process each table up to the numTables (default=1).

The properties are defined as: seesv -htmltable 0 "" "numTable 3 removePattern {some pattern} stripTags true"

Parse the input html file

• columns: Column names
  
• startPattern:
  
• endPattern:
  
• pattern: Row pattern. Use (...) to match columns
  

Harvest links in web page. This results in a 2 column dataset with fields: label,url

• pattern: regexp to match
  

Extract rows from the text

• comma separated header:
  
• chunk pattern:
  
• token pattern:
  

If you have an unstructured text file the -text command can extract out a set of rows and columns from it. The idea is you specify one regular expression (help) for finding a chunk of text in the file that holds the row values and another regular expression to extract the column values from the chunk.

Assume you have a file test.txt

  Hello there how are you
Person:  name: Jim age: 42
and some more text  
Person:  name: Joe age: 53
Person:  name: Sally age: 93
and more

We are going to extract the names and age with: seesv -text "name,age" "(?m)(?s).*?Person:(.*?$)" "name:(.*?)age: *(\d+)" The (?m)(?s) are directives to match multiple lines.
The (.*?$) in the first pattern is the group that is matched.
The second pattern: name:(.*?)age: *(\d+) has 2 groups, one for the name and one for the age.

Extract rows from the text

• comma separated header:
  
• token pattern:
  

If you have a row oriented but unstructured text file the -extractpattern command can be used to extract the columns from it. Specify a comma separated header and a regular expression (help) with groups that define the columns.

Assume you have a file test.txt

Person:  name: Joe age: 53
Person:  name: Sally age: 93

Extract the data with: seesv -extractpattern "name,age" "name:(.*)age:(.*)" -p file.txt

Tokenize the input from the pattern

• header: header1,header2...
  
• pattern:
  

Read data from the given database

• db: The database id (defined in the environment) values:property:seesv_dbs
  
• table: Comma separate list of tables to select from
  
• columns: Comma separated list of columns to select
  
• where: column1:expr:value;column2:expr:value;... e.g.: name:like:joe;age:>:60 Where expr is: =|<|>|<>|like|notlike
  
• properties: name space value properties. e.g., join col1,col2
  

The -sql command uses a number of RAMADDA properties or environment variables that define the databases that can be accessed. The command specifies a database as: seesv -sql <db> ...
e.g.:
seesv -sql test ...
Where for the given db there should be 2 environment variables:

export seesv_db_<db>_url=JDBC URL
export seesv_db_<db>_tables=List of tables that can be accessed

So for the db "test" in a Derby database:

export seesv_db_test_url=jdbc:derby:/Users/home/derby/db1;create=true;
export seesv_db_test_tables=test_table1,test_table2,test_table3

Generate an empty file with the given number of rows

• header: comma separated header
  
• values: comma separated values
  
• number_rows: Number of rows
  

For example using this: seesv -synthetic "name,value" "example name,example value" 5Gives:

name	value
example name	example value
example name	example value
example name	example value
example name	example value
example name	example value

Note: you don't have to specify all of the example values. For example, using this: seesv -synthetic "name,value" "example name,example value" 5 Gives:

name	value
<blank>	<blank>
<blank>	<blank>
<blank>	<blank>
<blank>	<blank>
<blank>	<blank>

Prune out the first N bytes

• bytes: Number of leading bytes to remove
  

Strip off the RAMADDA point header

Make the header proper capitalization

Clean up the header names

Use canonical names
This command converts the column names to a standardized format. Changing the names to lower case and converting punctuation, etc to "_".

Set XLS sheet #

• sheet: Sheet number
  

Concat the columns in one or more csv files

• *.csv:
  

For example, with the following 2 files: test1.csv:

name1	value1
foo	1
bar	2

test2.csv:

name2	value2	value3
xxx	1	3
yyy	2	7

Calling: seesv -cat test1.csv test2.csv Would result in an out file:

name1	value1	name2	value2	value3
foo	1	xxx	1	3
bar	1	yyy	2	7

Append the files, skipping the given rows in the latter files

• skip: Number of rows to skip
  
• files:
  • .csv
    

This concatenates the input files, passing through the first file whole but skipping over the given number of lines in the subsequent files. So with files: test1.csv:

name	value
foo	bar

test2.csv:

name	value
x	y

test3.csv:

name	value
l	m

Calling: seesv -append 1 test1.csv test2.csv test3.csv Results in:

name	value
foo	bar
x	y
l	m

Write out last N lines. include the header

• numlines: Number of lines to leave
  
• file:
  • .csv
    

Extract strings from the file name and add them as new columns

• pattern: Pattern to match
  
• columnnames: Comma separated list of column names
  

Skip number of raw lines.

• lines: How many raw lines to skip
  

Set max rows to process

• rows: Number of rows
  

Pass through rows that the columns each match the pattern

• columns: Column indices. Can include ranges, e.g. 0-5
  
• pattern: regexp or prefix with includes:s1,s2 to do substrings match
  

Pass through rows that don't match the pattern

• columns: Column indices. Can include ranges, e.g. 0-5
  
• pattern: regexp or prefix with includes:s1,s2 to do substrings match
  

Next N args specify a filter command followed by any change commands followed by an -endif.
For example, the below command would change the value of column2 for all rows that match the -pattern column1 string command. seesv -if -pattern column1 string -change column2 .* XXX -endif

• lines: Pass through and print out rawlines unprocesed
  

Filter out input lines that contain any of the strings

• filters: Comma separated list of strings to filter on
  

These patterns are applied to the raw lines that are read in before they are tokenized into columns.

Only pass thorough lines that have at least this number of columns. Specify blank to use the number of columns in the header

• min # columns:
  

Only pass through lines that have no more than this number of columns. Specify blank to use the number of columns in the header

• max # columns:
  

Remove or add values so each row has the number of columns

• number: use -1 to use the # of columns in the header
  

Only pass through anything if the data has the given columns

• columns: Column indices. Can include ranges, e.g. 0-5
  

Only pass through rows with number of columns passing the operator

• operator: <,<=,>,>=,=,!=
  
• number: Number of columns
  

Pass through rows that the columns each fuzzily match the pattern

• threshold: Score threshold 0-100. Default:85. Higher number better match
  
• columns: Column indices. Can include ranges, e.g. 0-5
  
• pattern:
  

Pass through rows that the length of the columns is greater than

• columns: Column indices. Can include ranges, e.g. 0-5
  
• length:
  

Pass through where the 2 columns have the same value

• column1:
  
• column2:
  

Pass through where the 2 columns don't have the same value

• column1:
  
• column2:
  

Pass through unique values

• columns: Column indices. Can include ranges, e.g. 0-5
  
• mode: What type of matching is done - exact (exact match) or clean (lower case and remove whitespace) or fuzzy:threshold (do fuzzy matching with threshold from 1: no similarity to 100: exact match. use fuzzy:? to print out values) values:exact,clean,fuzzy:threshold
  

The mode fuzzy:threshold uses the Fuzzywuzzy package which is based on the FuzzyWuzzy Python algorithm which uses the Levenshtein distance to calculate similarity between strings. You can view what the calculated threshold values are: seesv -unique column fuzzy:? This will print out the threshold values, e.g:

value:joey smith
        95 joe smith
value:sam jones
        44 joe smith
        42 joey smith
value:sam jones
        44 joe smith
        42 joey smith
value:sue washington
        43 joe smith
        42 joey smith
        35 sam jones
value:sue washington
        43 joe smith
        42 joey smith
        35 sam jones

Pass through duplicate values

• columns: Column indices. Can include ranges, e.g. 0-5
  

Pass through rows based on probablity

• probablity: 0-1 probability of passing through a row
  

For example, if you only want 5% of the data sampled do: seesv -sample 0.05 ...

Pass through the row that has the min value in the group of columns defined by the key column

• key column:
  
• value column:
  

Pass through the row that has the max value in the group of columns defined by the key column

• key column:
  
• value column:
  

Pass through rows that the column value equals the given value

• column:
  
• value:
  

Pass through rows that the column value does not equal the given value

• column:
  
• value:
  

Pass through rows that the column value is greater than the given value

• column:
  
• value:
  

Pass through rows that the column value is greater than or equals the given value

• column:
  
• value:
  

Pass through rows that the column value is less than the given value

• column:
  
• value:
  

Pass through rows that the column value is less than or equals the given value

• column:
  
• value:
  

Extract rows that are within the range

• column:
  
• min value:
  
• max value:
  

Extract rows that are not within the range

• column:
  
• min value:
  
• max value:
  

Extract rows that are between the given strings

• column:
  
• start string:
  
• end string:
  

For example, given the following data

name	value1	value2
jane	foo	1
jane	bar	2
john	doe	3
bill	doe	4
bill	doe	4

Calling: seesv -betweenstring name jane john Will give:

name	value1	value2
jane	foo	1
jane	bar	2
john	doe	3

Extract rows that are between the given strings

• column:
  
• start string:
  
• end string:
  

For example, given the following data

name	value1	value2
jane	foo	1
jane	bar	2
john	doe	3
bill	doe	4
bill	doe	4

Calling: seesv -notbetweenstring name jane john Will give:

name	value1	value2
bill	doe	4
bill	doe	4

One row in each group has to match

• column: key column
  
• value_column: Value column
  
• operator: values:=,!=,~,<,<=,>,>=
  
• value:
  

Pass through rows whose date is before the given date

• column:
  
• date:
  

Pass through rows whose date is after the given date

• column:
  
• date:
  

No more than count unique values

• column:
  
• count:
  

only include every row

• rows:
  1. of start rows to include
     
• skip: skip factor
  

Pass through rows that the columns with ID is in given file

• column: Column in the file
  
• file: The file
  
• column2: Column in main file
  

The -ifin command takes a file of values and will only pass through the row if the value in the row matches at least one of the values in the file. For example, say you have a file precincts.csv:

precinct

p1

p2

p3

And a file voters.csv:

name	voter precinct
jim	p1
joe	p1
sally	p7
...

If you only want voters whose precinct is in the precints.csv you would do: seesv -ifin precinct precincts.csv voter_precinct -p voters.csv The -ifnotin command does the opposite only passing through rows that are not in the file, e.g.: seesv -ifnotin precinct precincts.csv voter_precinct -p voters.csv

Pass through rows that the columns with ID is not in given file

• column: Column in the file
  
• file: The file
  
• column2: Column in main file
  

Pass through rows that the columns with ID begins with something in the given file

• pattern: Pattern template, e.g. ^${value}
  
• column: Column in the file
  
• file: The file
  
• column2: Column in main file
  

The -ifmatchesfile command takes a pattern template and a file of strings. For each line in the file it creates a regular expression pattern using the template by substituting the line into the pattern with the macro ${value}.

For example, say you have a file of addresses short.csv:

address	city	state
123 oak	boulder	co
345 main	denver	co
...

And your source data has a set of full addresses full.csv:

full address	city	state
123 oak unit #5
789 pine apt 3
345 main
...

And you only want to pass through rows in full.csv address begins with some address in the short.csv file. To do this call: seesv -ifmatchesfile "^${value}.*" address short.csv full_address -p fill.csv What this does is construct regexp patterns as:

^123 oak.*
^345 main.*
...

And then only passes through the rows in full.csv that match one of these patterns.

The command -ifnotmatchesfile does the opposite. For a row to pass through none of the patterns can match.

Pass through rows that the columns with ID does not begin with something in the given file

• pattern: Pattern template, e.g. ^${value}
  
• file: The file
  
• column2: Column in main file
  

Skip any line that matches the pattern

• pattern:
  

Skip number of processed rows.

• rows: How many rows to skip
  

Throw error if non-numeric

• columns: Column indices. Can include ranges, e.g. 0-5
  

Only include the given columns

• columns: Column indices. Can include ranges, e.g. 0-5
  

Don't include given columns

• columns: Column indices. Can include ranges, e.g. 0-5
  

Move columns to beginning

• columns: Column indices. Can include ranges, e.g. 0-5
  

Move columns to end

• columns: Column indices. Can include ranges, e.g. 0-5
  

Move columns before the given column

• column: Column to move before
  
• columns: Columns to move
  

Move columns after given column

• column: Column to move after
  
• columns: Columns to move
  

Remove the columns

• columns: Column indices. Can include ranges, e.g. 0-5
  

Drop rows

• rows: One or more rows. -1 to the end. e.g., 0-3,5,10,-1
  

Only include specified rows

• rows: one or more rows, -1 to the end
  

Move rows to the top that match the pattern

• columns: columns to match on
  
• pattern: Pattern
  

Move rows to the end of list that match the pattern

• columns: columns to match on
  
• pattern: Pattern
  

Copy column

• column:
  
• name:
  

Add new columns

• names: Comma separated list of new column names
  
• values: Comma separated list of new values
  

Insert new column values

• column: Column to insert before
  
• name: Name of new column
  
• values: Value to insert. Use ${row} to add the row index
  

Create a new column from the given columns

• columns: Column indices. Can include ranges, e.g. 0-5
  
• delimiter:
  
• name: Name of new colums
  

Concatenate multiple rows into a single row

• num_rows: Number of rows
  

For example, if you have this data:

name	value
joe	5
jill	2
jack	1
jim	7

Running: seesv -concatrows 2 Would give:

name	value	name	value
joe	5	jill	5
jack	1	jim	7

Combine columns with the delimiter. deleting columns

• columns: Column indices. Can include ranges, e.g. 0-5
  
• delimiter:
  
• column name: New column name
  

Combine columns with the delimiter

• columns: Column indices. Can include ranges, e.g. 0-5
  
• delimiter:
  
• column name: New column name
  

Apply operators to columns

• columns: Columns to merge
  
• name: New column(s) name
  
• operator: Operator values:average,min,max,count
  

Split the column

• column:
  
• delimiter: What to split on
  
• names: Comma separated new column names
  

Create a new column from the values in the given column

• keycol: Key column
  
• column:
  
• delimiter:
  
• name: new column name
  

Roll columns down into rows

• columns: Column indices. Can include ranges, e.g. 0-5
  

Shift columns over by count for given rows

• rows: Rows to apply to
  
• column: Column to start at
  
• count:
  

Slide columns down and over to append new rows to the bottom

• columns: Columns to move
  
• dest: Desc column to move to
  
• fill: Comma separated list of values to fill out the new row
  

Add a new cell at row/column

• row:
  
• column:
  
• value:
  

Delete cell at row/column

• row:
  
• column:
  

Clone each row N times

• count: Number of clonese
  

Only include specified rows

• skip: How many rows to skip
  
• count: How many rows to merge
  
• delimiter: How many rows to merge
  

For example, with the below data:

category	category	category
fruit	vegetable	meat
1	2	3
4	5	6

Running: seesv -appendrows 0 1 " - " Would give:

category - fruit	category - vegetable	category - meat
1	2	3
4	5	6

Merge rows

• rows: 2 or more rows
  
• delimiter:
  
• close:
  

Apply an operator to columns and merge rows

• keys: Key columns
  
• values: Value columns
  
• operator: Operator values:average,min,max,count
  

Rotate the data

Reverse the order of the rows except the header

Make new columns from data values

• column: Column to get new column header#
  
• value columns: Columns to get values from
  
• unique column: The unique value, e.g. date
  
• other columns: Other columns to include
  

Use values in header to make new row

• columns: Column indices. Can include ranges, e.g. 0-5
  
• header label:
  
• value label:
  

Make separate files based on value of column

• column:
  

Join the 2 files together

• key columns: key columns the file to join with
  
• value_columns: value columns
  
• file: File to join with
  
• source_columns: source key columns
  
• default_value: default value - can be a comma separated list of defaults
  

Join the 2 files together using fuzzy matching logic

• threshold: Score threshold 0-100. Default:85. Higher number better match
  
• key columns: Numeric column numbers of the file to join with
  
• value_columns: numeric columns of the values to join
  
• file: File to join with
  
• source_columns: source key columns
  
• default_value: default value
  

Make a cross product of 2 data files

• file: File to cross with
  

With the following data: letters.csv:

letter

a

b

c

d

number

1

2

3

The command: seesv -cross numbers.csv -p letters.csv Would give:

letter	number
a	1
a	2
a	3
b	1
b	2
b	3
c	1
c	2
c	3
d	1
d	2
d	3

Normalize the strings

• columns: Columns
  

Count number of unique values

• columns: Column indices. Can include ranges, e.g. 0-5
  

Make fields based on patterns

• column:
  
• pattern: e.g., "(field1:.*) (field2:.*) ..."
  

Make fields from key/value pairs, e.g. name1=value1 name2=value2 ...

• column:
  

Extract first N characters and create new column

• column:
  
• name: New column name
  
• number: Number of characters
  

Extract last N characters and create new column

• column:
  
• name: New column name
  
• number: Number of characters
  

Extract characters between the 2 indices

• column:
  
• name: New column name
  
• start: Start index
  
• end: End index
  

Extract column values from headings

• columns: Columns of headings
  
• names: Comma separated list of new column names
  
• pattern: Regexp to apply to header with () defining column values
  

Change columns

• columns: Column indices. Can include ranges, e.g. 0-5
  
• pattern: Regexp pattern. Help
  
• substitution string: use $1, $2, etc for pattern (...) matches
  

For example, with the below data:

label

xxxsome valueyyy

You can strip out the xxx...yyy and add a label: prefix with: seesv -change label "xxx(.*)yyy" "label:$1" Which would give:

label

label:some value

You can also specify a file that holds a number of patterns: seesv -change columns file:file.txt "" The file is of the form:

pattern1::replace_string1
pattern2::replace_string2  

e.g.:

(?i).*foo.*::foobar
.*value.*::new value

Change the values in the row/cols

• rows: Row indices. Can include ranges, e.g. 0-5
  
• columns: Column indices. Can include ranges, e.g. 0-5
  
• pattern: Regexp pattern. Help
  
• substitution string: use $1, $2, etc for pattern (...) matches
  

Replace

• columns: Column indices. Can include ranges, e.g. 0-5
  
• substitution string: use $1, $2, etc for pattern (...) matches
  use {value} for value
  

Write the value into the cells

• columns: Column indices. Can include ranges, e.g. 0-5
  
• rows: Row indices. Can include ranges, e.g. 0-5
  
• value:
  

Clean whitespace

• columns: Column indices. Can include ranges, e.g. 0-5
  

Look for the pattern in the header and apply the template to make a new column, template: '{1} {2} ...', use 'none' for column name for no header

• pattern: Regexp pattern. Help
  
• template:
  
• column label:
  

Write the value into the write col for rows that match the pattern

• column: match col #
  
• pattern: Regexp pattern. Help
  
• write column:
  
• value:
  

Copy column 2 to column 3 if all of the columns match the pattern

• columns: Column indices. Can include ranges, e.g. 0-5
  
• pattern: Regexp pattern. Help
  
• column1:
  
• column2:
  

Copy columns 1 to columns 2

• columns1:
  
• columns2:
  

Fill down with last non-null value

• columns: Column indices. Can include ranges, e.g. 0-5
  

Fill across with last non-null value

• columns: Column indices. Can include ranges, e.g. 0-5
  
• rows: One or more rows. -1 to the end. e.g., 0-3,5,10,-1
  

Set following cells to blank if the same as previous cell

• columns: Column indices. Can include ranges, e.g. 0-5
  

For example, given the following data

name	value1	value2
jane	foo	1
jane	bar	2
john	doe	3
bill	doe	4
bill	doe	4

Calling: seesv -unfill name,value1 Will give:

name	value1	value2
jane	foo	1
<blank>	bar	2
john	doe	3
bill	<blank>	4
<blank>	<blank>	4

Append prefix from the previous element to rows that match pattern

• column:
  
• pattern: Regexp pattern. Help
  
• delimiter:
  

Change case of column - type:lower,upper,proper,capitalize

• column:
  
• type: values:lower,upper,proper,capitalize
  

Convert the column(s) into IDS (lowercase, no space, a-z0-9_)

• columns: Column indices. Can include ranges, e.g. 0-5
  

Pad left with given character

• columns: Column indices. Can include ranges, e.g. 0-5
  
• character: Character to pad to
  
• length: Length
  

Pad right with given character

• columns: Column indices. Can include ranges, e.g. 0-5
  
• character: Character to pad to
  
• length: Length
  

Trim leading and trailing white space

• columns: Column indices. Can include ranges, e.g. 0-5
  

Trim leading and trailing quotes

• columns: Column indices. Can include ranges, e.g. 0-5
  

Limit the string size of the columns

• columns: Column indices. Can include ranges, e.g. 0-5
  
• size:
  

Add the text to the beginning of the file. use _nl_ to insert newlines

• text:
  

Add or remove columns to achieve the count

• count:
  
• pad string:
  

Add prefix to column

• column:
  
• prefix: String to use
  

Add suffix to column

• column:
  
• suffix:
  

Create a new column with the template

• column_name: New Column Name
  
• template: Template - use ${column_name} ...
  

Convert non ascii characters

• columns: Column indices. Can include ranges, e.g. 0-5
  
• substitution string: use $1, $2, etc for pattern (...) matches
  

Clean the phone number

• columns: Column indices. Can include ranges, e.g. 0-5
  

Format the phone number

• columns: Column indices. Can include ranges, e.g. 0-5
  

Add a true/false if the string is a mobile phone

• columns: Column indices. Can include ranges, e.g. 0-5
  

The -ismobile command uses one of two external APIs - https://numverify.com/ or https://numlookupapi.com. After making an account with either of these services set the corresponding environment variables:

export NUMLOOKUPAPI_API_KEY=
or
export NUMVERIFY_API_KEY=

This command is only available when running from the command line or if running within RAMADDA the user is a logged in user. We do this as the SeeSV service within RAMADDA can be run by any user.

A cache file, ismobile.txt, of past phone numbers will be created in the local working directory so this command can be run multiple times without resorting to the API calls for past phone numbers.

Send a text message - only for command line

• column: Phone number
  
• campaign: Campaign
  
• message: Message template
  

The -sms command uses https://www.twilio.com/ to send text messages. This command is only available when running from the command line or if running within RAMADDA the user is a logged in user. We do this as the SeeSV service within RAMADDA can be run by any user.

To enable this command create an account at Twilio and access your API keys. Add the following environment variables:

export TWILIO_ACCOUNT_SID=
export TWILIO_AUTH_TOKEN=
export TWILIO_PHONE=

This command takes a column ID which holds the phone number, a campaign identifier and a message template. The campaign identifier is used to create a list of phone numbers that a message has already been sent to. For example if you first run: seesv -sms phone some_campaign "Hi there"The SeeSV package will write out to a file <campaign>.sent.txt the list of phone numbers that the message was sent to. If the command is interrupted then rerun with the same campaign ID then those numbers that have had a message sent to will not have the message sent again.

Define Javascript (e.g., functions) to use later in the -func call

• javascript:
  

Apply the javascript function. Use _colname or _col#

• names: New column names
  
• javascript: javascript expression
  

Ensure that each column ends with the string

• column:
  
• string:
  

• columns: Column indices. Can include ranges, e.g. 0-5
  
• max length:
  
• suffix:
  

Extract text from column and make a new column

• column:
  
• pattern: Regexp pattern. Help
  
• replace with: use 'none' for no replacement
  
• name: new column name
  

Extract URL argument and make a new column

• column:
  
• argname: URL arg name
  

Extract text from HTML

• column: URL Column
  
• names: Comma separated list of new column names
  
• pattern: Regexp pattern. Help
  

Extract icon and description from input URL

• column: URL Column
  

Check for missing URL

• column: URL Column
  
• replace_with: Replace with
  

Encode the value for XML

• columns: Column indices. Can include ranges, e.g. 0-5
  

URL encode the columns

• columns: Column indices. Can include ranges, e.g. 0-5
  

URL decode the columns

• columns: Column indices. Can include ranges, e.g. 0-5
  

Change values in column to new values

• column:
  
• new column name:
  
• value: old_value new_value old_value new_value
  

Apply decimal format to the columns (see https://docs.oracle.com/javase/7/docs/api/java/text/DecimalFormat.html)

• columns: Column indices. Can include ranges, e.g. 0-5
  
• format: Decimal format e.g. '##0.00'
  

Read the id,value from file and substitute the value in the dest file col idx

• file: From csv file
  
• from id idx:
  
• from value idx:
  
• to idx:
  
• new col name:
  
• mode replace add:
  

Break apart column values and make new rows

• label1:
  
• label2:
  
• columns: Column indices. Can include ranges, e.g. 0-5
  

Parse out name and email

• columns: Column indices. Can include ranges, e.g. 0-5
  

Turn the header row into IDs (lowercase, no space, a-z0-9_)
For example, if you had the below data:

Some header label with some punctuation-+/	Some other header label
...

Running: seesv -makeids Gives

some_header_label_with_some_punctuation	some_other_header_label
...

Fake up data. See the docs at https://ramadda.org/repository/userguide/seesv.html#-faker

• what: firstname|lastname|fullname|etc
  
• columns: Columns to change. If none given then add the fake value
  

The -faker command uses the Java Faker package and takes any of the below commands:

firstname
fullname
lastname
name
namewithmiddle
prefix
suffix
title
username
address
city
country
state
stateabbr
streetname
timezone
zipcode
latitude
longitude
countrycode
boolean
asin
ean13
ean8
gtin13
gtin8
imei
isbn10
isbn13
isbngroup
isbngs1
isbnregistrant
color
department
material
price
productname
promotioncode
demonym
educationalattainment
maritalstatus
race
sex
bic
creditcard
iban
ssn
digit
digits:number_of_digits
numberbetween:first:last
randomdigit
randomdigitnotzero
randomDouble:maxNumberOfDecimals:min:max
randomnumber
cellphone
phonenumber
diseasename
hospitalname
medicinename
symptoms

Hand edit a column (command line only). ESC-stop, BLANK-skip

• column: key column
  

The edit command is used from the command line and will prompt the user for a new column value. If the value has already been seen then the prior new value is used. Enter <blank> to skip the given row. Enter ESCAPE to stop prompting for new values.

Make a message digest of the column values

• columns: Column indices. Can include ranges, e.g. 0-5
  
• type: values:MD5,SHA-1,SHA-256,SHA-512,SHA3-256,SHA3-512,
  

Base 64 Encode

• columns: Column indices. Can include ranges, e.g. 0-5
  

Base 64 Decode

• columns: Column indices. Can include ranges, e.g. 0-5
  

Rot 13

• columns: Column indices. Can include ranges, e.g. 0-5
  

Encrypt using AES with SHA-256 key

• columns: Column indices. Can include ranges, e.g. 0-5
  
• password:
  

Encrypt using AES with SHA-256 key

• columns: Column indices. Can include ranges, e.g. 0-5
  
• password:
  

Add a UUID field

Add 1,2,3... as column

Add 'A','B', ... as column

Generate a soundex code

• columns: Column indices. Can include ranges, e.g. 0-5
  

Add a description from Wikipedia

• columns: Search string columns
  
• suffix: Text to add after
  

This uses the API at https://en.wikipedia.org/w/api.php to get the description of a wikipedia page that matches the string provided by the given columns.

Do a Bing image Search for an image

• columns: Column indices. Can include ranges, e.g. 0-5
  
• suffix: Text to add after
  

Download the URL and embed the image contents

• url column:
  

Fetch the URL and embed the contents

• name: Name of new column
  
• ignore_errors: Ignore Errors e.g., true or false
  
• url: URL template, e.g., https://foo.com/${column_name}
  

Search for an image with the query column text if the given image column is blank. Add the given suffix to the search.

• querycolumn:
  
• suffix:
  
• imagecolumn:
  

Download the URL

• column: Column that holds the URL
  
• suffix: File suffix
  

Figure out the gender of the name in the column

• columns: Column indices. Can include ranges, e.g. 0-5
  

Specify one or more date formats for parsing

• format: e.g. yyyy-MM-dd HH:mm:ss. Use semi-colon separated formats for multiples. Help
  
• timezone:
  

Specify date format for formatting

• format: e.g. yyyy-MM-dd HH:mm:ss. Help
  
• timezone:
  

Convert date

• columns: Columns to convert
  

Add date

• date_column: Date Column
  
• value: Value Column
  
• value_type: Value type - millisecond,second,minute,hour,hour_of_day,week,month,year
  

Clear date components

• date_column: Date Column
  
• component: Date component values:millisecond,second,minute,hour_of_day,day_of_month,month
  

Extract date

• date column:
  
• what: What to extract, e.g., year, month, day_of_week, etc values:era,year,month,day_of_month,day_of_week,week_of_month, day_of_week_in_month,am_pm,hour,hour_of_day, minute,second,millisecond,days_in_year, hours_in_year, minutes_in_year,seconds_in_year
  

Format date

• columns: Column indices. Can include ranges, e.g. 0-5
  

Format the date offset, e.g. the hours in year

• column:
  
• what: What type of offset, e.g., year, month, day_of_week, etc values:days_in_year, hours_in_year, minutes_in_year,seconds_in_year
  

Calculate elapsed time (ms) between rows

• column:
  

Convert milliseconds to

• column:
  
• to: seconds|hours|days|weeks|months|years
  

Pass through rows whose date is the latest in the group of rows defined by the key column

• columns: Key columns
  
• column: Date column
  
• format: Date Format, e.g. yyyy-MM-dd
  

Calculate elapsed time between columns column1-column2

• column1: Column 1
  
• column2: Column 2
  
• unit: Unit-milliseconds,seconds,minutes,hours,days values:milliseconds,seconds,minutes,hours,days
  

add a true/false column comparing the date values

• column1:
  
• column2:
  
• operator: <,<=,=,!=,>=,> values:<,<=,=,!=,>=,>
  

Set value={value+delta1}*scale+delta2

• columns: Column indices. Can include ranges, e.g. 0-5
  
• delta1:
  
• scale:
  
• delta2:
  

The -scale command adds offset and seesv -scale <column id> <offset1> <scale> <offset2>Where the result value, v2:

  v2 = (value+offset1)*scale + offset

You can specify random values for any of the values with:

#Generate a random number between 0-1
random:

#Generate a random number between 10 and 11
random:10

#Generate a random number between 10 and 30
random:10:30

For example, if you had a latitude value that you wanted to obfuscate by adding some random value between 1 and 2 you would do:

  -scale latitude random:1:2 1 0

Try to parse as number

• columns: Column indices. Can include ranges, e.g. 0-5
  

Add row values

• label:
  
• start:
  
• step:
  

Round decimals

• columns: Column indices. Can include ranges, e.g. 0-5
  
• num_decimals: how many decimals to round to
  

fuzz the number. if num_places less than zero than that is the # of decimals. else that is the lower digits to fuzz out

• columns: Column indices. Can include ranges, e.g. 0-5
  
• num_places: how many places to round to. use <=0 for decimals
  
• num_random_digits: how many random digits
  

The -fuzz command works to anonymize numeric values.

The num_random_digits is used to generate a number with those random digits.

If the num_places argument is <= zero then that is the number of decimal places the number is rounded to. If there are fewer than num_places decimals than those decimals are converted to "0". The randomized digits are then appended to the number. For example with R = random digit: seesv -fuzz <columns> -2 3 Gives:

12345.6789 -> 12345.67RR
145.678 -> 145.67RR
9.0 -> 9.RR

If num_places > 0 then the number is converted to an integer and the num_places least significant digits are converted to 0. If the number of digits is < num_places then the number is converted to 0. The randomized digits are then addedto the number. For example with R = random digit: seesv -fuzz 2 3 Gives:

12345.678 -> 123RR
145.678 -> 1RR
1.678 -> RR

Set the max value

• columns: Column indices. Can include ranges, e.g. 0-5
  
• value: Value
  

Set the min value

• columns: Column indices. Can include ranges, e.g. 0-5
  
• value: Value
  

Add column that is the delta from the previous step

• key columns:
  
• columns: Column indices. Can include ranges, e.g. 0-5
  

Make a running sum of the column values

• columns: Column indices. Can include ranges, e.g. 0-5
  

Make counter field that is incremented everytime the value column decreases

• column: The value column
  
• name: Name of counter column
  

Apply the operator to the given columns and create new one

• columns: Columns
  
• new col name:
  
• operator: Operator:+,-,*,/,%,average values:+,-,*,/,%,average
  

Add a true/false column comparing the values

• column1:
  
• column2:
  
• operator: <,<=,=,!=,>=,> values:<,<=,=,!=,>=,>
  

Round the values

• columns: Column indices. Can include ranges, e.g. 0-5
  

Make absolute values

• columns: Column indices. Can include ranges, e.g. 0-5
  

Clip the number to within the range

• columns: Column indices. Can include ranges, e.g. 0-5
  
• min:
  
• max:
  

make random value

• column name:
  
• minrange: Minimum range (e.g. 0)
  
• maxrange: Maximum range (e.g. 1)
  

Add true if the column starts with an even number

• columns: Column indices. Can include ranges, e.g. 0-5
  

Sum values keying on key column value. If no value columns specified then do a count

• key columns:
  
• value columns:
  
• carry over columns:
  

Make a pivot table

• key columns: Columns to key on
  
• column columns: The columns the values of which are used to make the new columns in the result
  
• value column: The value column
  
• operator: The operator to apply - count,sum,average,min,max
  

For example, with the input file:

Region	Date	Units	Sales
West	2016	1	11.00
South	2016	8	96.00
West	2016	2	26.00
North	2016	7	84.00
North	2016	8	104.00
South	2016	2	22.00

Running: seesv -pivot region date sales count -p input.csv Gives:

Region	2016	2017	2018	2019
West	15.0	51.0	49.0	42.0
South	25.0	23.0	31.0	14.0
North	15.0	27.0	56.0	39.0
East	10.0	24.0	18.0	13.0

You can also specify multiple operators: seesv -pivot region date sales count,min,max -p input.csv Gives:

Region	2016 - count	2016 - min	2016 - max	2017 - count	2017 - min	2017 - max	2018 - count	2018 - min	2018 - max	2019 - count	2019 - min	2019 - max
West	15.0	11.0	96.0	51.0	11.0	104.0	49.0	11.0	112.0	42.0	13.0	120.0
South	25.0	12.0	96.0	23.0	11.0	104.0	31.0	11.0	120.0	14.0	12.0	112.0
North	15.0	12.0	104.0	27.0	12.0	96.0	56.0	11.0	120.0	39.0	13.0	120.0
East	10.0	24.0	96.0	24.0	11.0	104.0	18.0	11.0	112.0	13.0	12.0	105.0

count/sum/average/min/max values keying on key column value. If no value columns specified then do a count

• key columns: Columns to key on
  
• value columns: Columns to apply operators on
  
• carry over columns: Extra columns to include
  
• ops: any of count,sum,average,min,max
  

For example, with the below data, if you wanted to summarize the total units by region:

Region	Color	Units	Sales
West	Red	1	11.00
South	Blue	8	96.00
West	Green	2	26.00
North	Blue	7	84.00
North	Green	8	104.00
South	Red	2	22.00
East	Blue	5	60.00
West	Green	2	26.00

You would do: seesv -summary region units "" sum -p summary.csv Which gives:

Region	Units sum
West	683.0
South	399.0
North	672.0
East	294.0

If you wanted to get the sum and average of the sales: seesv -summary region sales "" sum,avg -p summary.csv You'd get:

Region	Sales sum	Sales avg
West	8873.0	56.51592356687898
South	5044.0	54.236559139784944
North	8709.0	63.56934306569343
East	3730.0	57.38461538461539

The carry over column just gets the first value for the key column. So, if you wanted to include the color column do: seesv -summary region sales color sum,avg -p summary.csv Which gives:

Region	Sales sum	Sales avg	Color
West	8873.0	56.51592356687898	Red
South	5044.0	54.236559139784944	Blue
North	8709.0	63.56934306569343	Blue
East	3730.0	57.38461538461539	Blue

Make a histogram with the given column and bins

• column: The column
  
• bins: Comma separated set of bin values
  
• value columns: Extra columns to sum up
  
• ops: ops to apply to extra columns - any of count,sum,average,min,max
  

The bins value is a comma separated list of the numeric ranges that define how to bin the data. e.g., for a set of age ranges one can specify:

bins="18,30,40,50,60,70"

This results in bins of:

0-18
18-30
30-40
40-50
etc.

You can also specify a range with an optional step value. For example this:

bins="18-30" 

Will give you the bins:

18-19
19-20
20-21
...
29-30
30-infinity

Or you can specify a step

bins="18-30:2" 

Will give you the bins:

18-20
20-22
22-24
...

Add columns together. Replace with their percentage

• columns: Columns to add
  

Calculate percent increase

• columns: Column indices. Can include ranges, e.g. 0-5
  
• how far back:
  

Difference from previous value

• columns: Column indices. Can include ranges, e.g. 0-5
  
• how far back (default 1):
  

Calculate a moving average

• columns: Columns
  
• period:
  
• label:
  

Create a new column with the (string) ranges where the value falls in

• columns: Column indices. Can include ranges, e.g. 0-5
  
• name: New column name
  
• start: Numeric start of range
  
• size: Numeric size of range
  

Convert suffixed values (e.g., 2 MB) into the number

• unit: values:binary,metric
  
• columns: Column indices. Can include ranges, e.g. 0-5
  

And values

• name: New column name
  
• columns: Column indices. Can include ranges, e.g. 0-5
  

Or values

• name: New column name
  
• columns: Column indices. Can include ranges, e.g. 0-5
  

Not value

• name: New column name
  
• column:
  

Check that the values are numbers

• columns: Column indices. Can include ranges, e.g. 0-5
  
• what: How strict values:strict,ramadda
  

Geocode using given columns

• columns: Address columns
  
• prefix: optional prefix e.g., state: or county: or country:
  
• suffix:
  

You can use Google or geocode.io for geocoding by setting the following environment variables. Else, this defaults to using US Census geocode API.

To use Google geocoding set the environment variable:

GOOGLE_API_KEY=...

To use geocod.io set the environment variable:

GEOCIDEIO_API_KEY=...

For here.com set:

HERE_API_KEY=

To geocode, e.g., with the following file:

city	state
Boulder	CO
Laramie	WY
Buffalo	NY

Run: seesv -geocode city,state "" "" ... This concatenates the city and state field, e.g., "Boulder CO" and calls the API to geolocate.

To use internal location tables set the prefix argument to one of: seesv -geocode columns "city:" "" ...
seesv -geocode columns "county:" "" ...
seesv -geocode columns "state:" "" ...
seesv -geocode columns "zip:" "" ...
seesv -geocode columns "country:" "" ...
The suffix gets appended to the query string. For example if you had a file with just cities in a state, e.g.:

city

Boulder

Denver

Grand Junction

Pueblo

You can geocode with: seesv -geocode city "" "CO"

Geocode if needed

• columns: Address columns
  
• prefix: optional prefix e.g., state: or county: or country:
  
• suffix:
  
• latitude: latitude column
  
• longitude: longitude column
  

Geocode for import into RAMADDA's DB. The lat/lon is one semi-colon delimited column

• columns: columns
  
• prefix: optional prefix e.g., state: or county: or country:
  
• suffix:
  

Geocode with file

• column: key column
  
• csv file: File to get lat/lon from
  
• key idx:
  
• lat idx:
  
• lon idx:
  

This assumes you have a separate file that contains a key value and latitude/longitude and the source file has a key value. For example, you might have a list of precincts, e.g.:

precinct	latitude	longitude
4171107648	40.197055	-105.100351
2171207308	39.996986	-105.084938
2173307410	40.039912	-105.081393

The source file is a list of voters, e.g.:

name	precinct
John Doe	4171107648
Jane Doe	4171107648
Jim Smith	2171207308

To add the lat/lon: seesv -geocodejoin precinct precincts.csv precinct latitude longitude

Geocode within bounds

• north:
  
• west:
  
• south:
  
• east:
  

Decode latlon

• columns: Lat or Lon column
  

This converts a column that is of the form degrees minutes seconds into it decimal degrees equivalent. For example, with this data:

latitude	longitude
40° 1' 12 N	107° 40' 10 W
40:1:12	107:40:10
40:1:12 N	107:40:10 W
40:1	107:40

Running the commands: seesv -decodelatlon latitude -decodelatlon longitude Gives:

latitude	longitude
40.0	-107.0
40.019999999999996	107.66944444444445
40.019999999999996	-107.66944444444445
40.016666666666666	107.66666666666667

If you wanted to have less precision then call: seesv -decodelatlon latitude -decodelatlon longitude -decimals latitude,longitude 5 -p test.csv Which gives:

latitude	longitude
40.0	-107.0
40.02	107.66944
40.02	-107.66944
40.01667	107.66667

Get address from lat/lon

• latitude: latitude column
  
• latitude: latitude column
  

This function takes a latitude and longitude column and does an address lookup using either geocod.io or here.com apis. See above for the API keys that need to be set.

Add state name from state ID

• state_column: State ID column
  

Look up location name

• lookup: ('counties' or 'states' or 'countries' or 'timezones')
  
• fields: fields in shapefile
  
• latitude: Latitude column
  
• longitude: Longitude column
  

This uses one of the shapefiles provided by RAMADDA and finds the name of the feature that contains the given lat/lon. For example if you had the following data file:

name	latitude	longitude
City 1	40.197055	-105.100351
City 2	39.996986	-105.084938
City 3	40.039912	-105.081393

You can add the country and timezone with: seesv -geoname countries latitude longitude -geoname timezones latitude longitude

Check for containment

• lookup: ('counties' or 'states' or 'countries' or 'timezones')
  
• name: new column name
  
• latitude: Latitude column
  
• longitude: Longitude column
  

Look up elevation(using 1/3 arc-second DEM)

• latitude: Latitude column
  
• longitude: Longitude column
  

This uses the web service at https://nationalmap.gov/epqs/pqs.php to look up the elevation for the given lat/lon. For example if you had the following data file:

name	latitude	longitude
City 1	40.197055	-105.100351
City 2	39.996986	-105.084938
City 3	40.039912	-105.081393

You can add the elevation with: seesv -geoname countries latitude longitude -geoname timezones latitude longitude

Convert x/y to lon/lat

• columns: x and y columns
  

Add the state's region

• columns: Columns with state name or abbrev.
  

This does a lookup from an internal table of the regions that maps state name (or abbrev) to region:

Alaska=West
AK=West
Alabama=South
AL=South
Arkansas=South
...

So with the following data:

name
Wyoming
California
Colorado

The region is added with: seesv -region name

Add in population from address

• columns: Column indices. Can include ranges, e.g. 0-5
  
• prefix: e.g., state: or county: or city:
  
• suffix:
  

Look up neighborhood for a given location

• latitude: Latitude column
  
• longitude: Longitude column
  
• default: Default value
  

This command uses the API from https://www.precisely.com/ to look up the neighborhood name from a given lat/lon. To use this get and API key from Precisely and set the environment variable:

PRECISELY_API_KEY=

Call: seesv -neighborhood latitude_column longitude_column

Apply the commands to each of the columns

• columns: Columns to expand with
  
• commands: Commands. Use the macro ${column}. End with -endapply
  

• columns: Column to sort on
  
• direction: Direction - up or down values:up,down
  
• how: How to sort - string, length, date, extract (number) values:string,number,length,date,extract
  

Show count

Set a field alias

• name: Name
  
• alias: Alias
  

The alias command can be used to make your commands easier to deal with if you have long field names. For example, say you have data like:

some_really_long_field_name

1

2

You can use the alias command to provide a short name, e.g.: seesv -alias some_really_long_field_name name -columns name ...

Define a macro value for later use

• name: name
  
• value: Value
  

Extract a macro value from a filename

• name: Macro name
  
• pattern: Regexp pattern. Help
  

Change the line

• from: From pattern
  
• to: To string
  

Change input text

• from: From pattern
  
• to: To string
  

Crop last part of string after any of the patterns

• columns: Column indices. Can include ranges, e.g. 0-5
  
• patterns: Comma separated list of patterns
  

Be strict on columns. any rows that are not the size of the other rows are dropped

Be strict on columns. any rows that are not the size of the other rows are shown

Set a property

• property: values:position
  
• value: start, end, etc
  

Go easy on missing columns

Verify that all of the rows have the same # of columns

Execute the external program

• program_id: matches with seesv.ext.<program_id>=/path
  

The -ext command allows one to integrate external programs into the processing flow. seesv -ext <id> <arguments> The id is used to look up an environment variable (or RAMADDA runtime property) of the form:

seesv_ext_<id>=/path/to/executable

Any command line arguments given up to but not including an argument that equals "-" are passed in to the executable. The executable is called with the given arguments. Each row that is processed is converted into a CSV string which is passed to the external command on the stdin. A CSV text string is then read from the external command. This line is then passed on in the processing flow. If you want to skip a line the return the string "_null_".

Execute the external program for every line

• program_id: matches with seesv.ext.<program_id>=/path
  

The -exec command allows one to integrate external programs into the processing flow. seesv -exec <id> <arguments> The id is used to look up an environment variable (or RAMADDA runtime property) of the form:

seesv_exec_<id>=/path/to/executable

Any command line arguments given up to but not including an argument that equals "-" are passed in to the executable. For every row the executable is called with the given arguments. Each of the arguments can contain any number of macros of the form:

seesv  -exec <id> ${url}  ${name}

The macros get replaced with the corresponding values in the given row.

So for example, if you have a list of urls that you want to fetch using cURL for set the environment variable:

export seesv_exec_curl=/usr/bin/curl    

Then assuming you have the file:

url
url1.html
url2.html

Call the command

seesv  -exec curl ${url}   - -p

This produces a new file:

url,result
url1.html,contents of url1
url2.html,contents of url2