A collection of usefull functions for the APL programmer.
apk_kbd is a workspace to add an additional shift-key to your keyboard so that you can type APL functions and operators. Once activated use the new shift-key with other keys to generate apl’s functions and operators. To see which key to use type the command )KEYBOARD for a keyboard diagram
Turn the new shift key on. shift_key is one of
Return the keyboard to its original state.
When choosing which shift key to use be wary of what other programs might be using it. Some window managers will use the Windows key for its own purposes.
arg_parser parses command line arguments that follow posix long-name options. That is a double dash followed by the argument and, optionally, followed by an equals sign and a value. That is –option[=value].
arg_parser presents an application programming interface:
For instance:
po←ap∆init 'Bill''s application' po←po ap∆new_option 'name' 'first' 'type' 'string' 'help' 'This application is helpless..' po←po ap∆new_option 'name' 'second' 'type' 'boolean' 'help' 'Do you agree that it is helpless?' po←ap∆parse po ⍎(ap∆has_errors po)/'⍞←ap∆get_errors po ◊ →0' ⍎(ap∆has_help po)/'⍞←ap∆help po ◊ →0' First←po ap∇get_option_value 'first' Second←po ap∆get_option_value 'second'
arg_parser uses both the utl and lex workspace. As shipped the arg_parser.apl file copies these utilities into its workspace. APL may not be able to find these utilities at run time. We recomend copying all three apl files into your script file to avoid any confusion.
Function returns a new parser object
Function adds a new option to the parser instance. The object defined in the right argument is a list of name value pairs for the following names:
Function examines the command line, confirms that command line options have been defined with new_option; that string type options have value and that boolen options do not. It then saves the given values.
Function sets errors true when it detects errors and sets help true if the operator included -help on the command line.
Function returns the value of an option. If a string option was not on the command line, ⍬ will be returned. If a boolean option was one the command line 1 (True) will be returned, if not, 0 will be returned.
Functions returns whether or not an error was found on the command line.
Function returns an error message about each error found.
Function returns whether or not help was requested.
Function returns a help message showing each defined object.
This library contains functions to perform unit testing. There are five basic functions, assert∆toScreen, assert∆return, assert∆nil∆toScreen, assert∆nil∆return and assert∆err. These functions will execute their right argument (the test) and compare its results to the left argument.
There are two functions for the environment, assert∆setup and assert∆cleanup.
Execute a list of commands to setup for testing
Execute a list of commands to clean up after testing
Prints a message to the screen indicating whether the test succeeded or failed.
returns a Boolean value indicating whether the test succeeded or failed.
Function to test a function without a return value. Such a function must have some side effects as it has no actual effect. One must write a function to test for the side effects and return True or False. Supply the name of that function as a character string.
Like assert∆nil∆toScreen, returns true or false rather than cluttering up your screen.
A simple (minded) function to display a message
A Function to test that an error has occurred. Right now this is empty function. When I get a bright idea on how to do it I will.
A helper function to recursively evaluate nested test results. Navigates through the complexity of nested array to define what equals actually means.
cfg parses Windows style configuration files (ini files). These are text files, usually with a suffix of ini, used by various programs to store configuration information.
Ini files are broken down into sections of name–value pairs. This workspaces stores this information in a lexicon of lexicons (see workspaces lex, lex1 and cl). That is each section is separate item in the first level lexicon and each name–value pair in the section is a separate item in the second level of the lexicon.
Reads an ini file and return a two level lexicon. An optional left argument supplies the character used to begin comment lines in the file.
This workspace implements Lillian dating, that is storing dates as the number of days from 10/15/1582. It was proposed by IBM in 1986, and named after Aloysius Lilius who devised the Gregorian Calendar.
Lillian dating simplifies date arithmetic as any date is stored as a simple integer.
This workspace contains the following functions:
Returns a Lillian date for a three element vector of year,month,day.
Returns a vector of year,month,day from a Lilian date.
Returns a vector of integers for year, month and day. If date∆parse is unable to parse the string it will return an error message.
The locale is a lexicon of the following key-value pairs
The workspace also has functions to return dates as text.
Returns string of three numbers separated by ’/’. The order of these numbers is determined from values in locale of the keys year_pos, month_pos and day_pos.
Returns a string of six numbers: year, month, day, hour, minute, and second followed by either ’am’ or ’pm’. Like date∆fmt∆3numbers the order of year, month and day are determined by the locale.
Returns a scheduled date suitable for use with emacs org-mode.
A locale lexicon for US usage
An 2 12 shaped array where line one is the days of each month in a leap year and line two the days of each month in a normal year.
A lexicon of two name-value pairs. ’Year 0’ is 1200. That is the previous year divisible by 4 that is not a leap year. (The essence of the Gregorian calendar reform).
’Pre lillian’ is the number of days from 1199 12 31 to 1582 10 15.
Its best not to ask why this is needed.
This workspace provides an incomplete implementation of w3.com’s Document Object Model (DOM). The DOM creates and manipulates a graph database from an xml file. The specification leans heavily on object oriented programing constructs.
This implementation provides a functional programming model with a function naming scheme to identify the objects in the specification. So that functions in the dom∆node family are methods specified for the node object and dom∆document functions are methods specified for the document object. Creation methods require a left argument of the name of the document variable, an idea not fully implemented in this version.
The graph database design departs from the DOM in that children of a node are stored in an apl vector and the node methods firstChild, lastChild and nextSibling have not yet been implemented.
Traversal of the graph is best illustrated by the function dom∆node∆toxml. That function recursively traverses the graph returning the variable xml, viz.:
xml←xml,∊dom∆node∆toxml ¨ dom∆node∆children node
To build a DOM use dom∆parse:
)copy 5 FILE_IO
dv ← 'dv' dom∆parse FIO∆read_file 'ADom.xml'
To look at a DOM graph use dom∆node∆toxml:
⍞←dom∆node∆toxml node
Export is an apl workspace to export an array to a delimited text file.
Function to create a file delimited by line feeds and tabs from an array of two dimensions.
Function to create a file delimited by line feeds and tabs from an nested array of vectors.
The finance workspace provides functions useful in understanding the cash flow and cash requirements of an enterprise and for planning and managing that cash flow.
These functions work on a single cash payment. Each function expects a right argument vector of amount, interest rate, and number of periods.
Calculate the future value of a single sum. All arguments are made in a single right-argument vector of cash invested, interest rate per period, and number of periods.
Interest rates are generally quoted at an annual rate ignoring the effects of compounding. Therefore $100 invested at 12% per annum and compounded monthly would yield $112.68 from entering
fin∆compoundValue 100, (.12÷12) 12 112.68
Function calculates the present value of a single sum payable in n periods. The right argument is assembled as with fin∆compoundValue. This is the reverse of fin∆compound value:
fin∆presentValue 112.68 .01 12 100
These functions work on a flow of cash. For instance a mortgage (called an annuity here) is usually a loan of a specific sum (negative cash flow) followed by monthly payments and a fixed amount (positive cash flow). For these function the following datum appear
The periodic payment
The interest per period
The number of periods.
Function calculates the present value of an annuity, that is the amount of a loan today in exchange for a payment in each of N periods. The right argument is a vector of the payment, interest rate, and number of periods, viz.:
fin∆presentValueAnnuity 100 .01 360 9721.83
Function calculates the future value of an annuity. That is the amount in a savings account after n periods of depositing the same amount.
fin∆compoundAnnuity 100 .01 360 352991.38
Function calculates the net present value of a series of cash receipts and disbursements. The left argument is the interest rate and the right a vector of cash flow items. Conventionally, the receipts are positive and disbursement negative.
The theory is that a firm has a cost of capital, that is an average rate of both the liabilities and equity. An investment is evaluated using that rate and the expected cash flow from the investment. This calculation can be made directly from that data. Some like internal rate of return (see fin∆irr next) are more difficult. Usually one must take the nth root of a number and therefore one has n possible solutions. The finance workspace uses a converging iteration to find one of those solutions.
.1 fin∆netPresentValue ¯100000 10000 11000 12000 14000 15000, 10⍴18000 13408.07
Internal rate of return. That is the interest rate implied by a vector of cash flows. This return is calculated iteratively using the result of the last rate of return for the current calculation. One must supply a guess to start the process.
.1 fin∆irr ¯100000 10000 11000 12000 14000 15000, 10⍴18000 0.1222471688
Dateflows are data structures that facilitate compound interest calculations where cash flows are not annuities or simple amounts. Each cash transfer, to or from, the investment is stored in the dateflow array with it dollar amount. The finance–dateflow functions here assume that the date flow is complete, that is every cash transfer to and from the investment is included.
With this data several things can be done to evaluate the investment. Fin∆df∆irr will compute the internal rate of return; fin∆df∆net_pv will compute the net present value of the investment using the supplied interest rate; and fin∆df∆amortization table will prepare a worksheet computing the future value of the investment using the supplied interest rate.
Dateflows may be built using fin∆df∆init and fin∆df∆add. Fin∆df∆show displays the supplied dateflow.
For example, Apple common stock had a closing price of 66.82 per share on November 30, 2019. One year later it was 121.01. During that year Apple payed four dividends of 0.19 ((2020/02/07); 0.82 (2020/05/07); 0.21 (2020/08/07); and 0.19 (2020/11/07). We construct a dateflow as follows:
apple←fin∆df∆init (date∆lillian 2019 11 30),¯66.82
apple←apple fin∆df∆add (date∆lillian 2020 02 07) .19
apple←apple fin∆df∆add (date∆lillian 2020 05 07) .82
apple←apple fin∆df∆add (date∆lillian 2020 08 07) .21
apple←apple fin∆df∆add (date∆lillian 2020 11 07) .19
apple←apple fin∆df∆add (date∆lillian 2020 11 30) 121.01
fin∆df∆show apple
2019/11/30 (66.82)
2020/02/07 0.19
2020/05/07 0.82
2020/08/07 0.21
2020/11/07 0.19
2020/11/30 121.01
with this date flow we compute the internal rate of return as follows:
fin∆df∆irr apple .5 0.6084386096
Where apple is our date flow variable and .5 is an estimated interest rate. Any gues will do, the closer to the actuall return, the faster the calculation.
Returns a dateflow. item is a vector of lillian date and amount. Generally this item is the amount of the initial ivestment and is negative.
Returns two date flows. Items in dateflow1 not in dateflow2 will be added to dateflow2. For items where the dates in dateflow1 match an item in dateflow2, dateflow2’s amount will be increased by dateflow1’s amount.
Returns dateflow1 less dateflow2. That is each amount in dateflow2 is negated and then added to dateflow1.
The amounts in dateflow will be multiplied by the given amount
Function will display a dateflow, converting the lillian dates to yyyy/mm/dd.
Function returns true if the right argument is a dateflow.
Function returns the net present value of the date flow. The right argument is two items: First the dateflow and second the interest rate.
Function returns the internal rate of return of a dateflow. The right argument is first the date flow and second an esimtate of the internal rate of return. The closer the estimate is to the actual rate, the quicker the calculation.
Function returns an amortization table. That is a table showing the initial investment, assumed interest earned, each cash transfer and a closing balance after each transfer as you would see in a pass book for a savings account.
The fmt workspace provides a partial implementation of STSC’s ⎕fmt system function. The function will format numeric data in various ways and provides an alternative to providing example formats to ⍕.
fmt takes as its left argument a character string that describes, column by column, how to render the data in an array, or number by number in a vector. This argument, we’re calling the format string, is made up of one or more phrases separated by commas. Each phrase has one character that controls basic formatting. ’F’ for fixed point, ’I’ for integers, ’E’ for scientific notation (exponential) or ’A’ for characters (alpha-numeric) are examples. There are more, however they are not yet implemented in this workspace.
Each phrase in the the format string is separated from other phrases by a comma. The phrase may contain a number for how many times the phase repeats; Modifiers with their arguments; the phrase type (E F I E A and some other letters); the width of the resulting column; and the number of decimal places or number of significant digits. Each type expects different modifiers.
For example, the phrase 3M/(/N/)/CF14.2 would be parsed as follows:
Note the delimiters around the text argument to the M and N modifiers. The valid text delimiters will be covered in See Delimiting text decoration.
Possible modifiers vary by phrase type. See See Fixed point; See Integers; See Exponential; and See A, Formating descriptive (text) columns.
Fixed point formatting is used for numbers that may be made up of a whole number and a fractional number. The fixed point (type F) phrase must contain both the width and precision, or number of decimal places.e.g.,
F14.2
Calls for a column 14 characters wide and two places to the right of the decimal.
Fixed point phrases may also contain modifiers and should look like this:
rmFw.d
Where:
Valid modifiers for Fixed point phrases are:
Multiply the number by 10 raised to the ith power.
Start each negative number with text. To differentiate positive and negative numbers one or more of M, N, P, or Q must be used.
End each negative number with text.
Start each positive number with text.
End each positive number with text.
The number will be padded both left and right with zeros. If M, N, P or Q is used the amount of padding will be reduced to allow room for the decorators.
Integer formatting is used for whole numbers. The decimal point will not be displayed. The fields width is required. e.g.,
I10
calls for a column ten characters wide.
Integer phrases may also contain the repetition count, and modifiers and should look like this:
rmIw
Where
Valid Integer modifiers are:
Multiply the number by 10 raised to the ith power.
Start each negative number with text. To differentiate positive and negative numbers one or more the this and the following three (N P and Q) must be used.
End each negative number with text.
Start each positive number with text.
End each positive number with text.
The number will be padded both left and right with zeros. If M, N, P or Q is used the amount of padding will be reduced to allow room for the decorators.
Exponential or scientific notation displays each number as a number between 0 and 10 and the exponent of 10 for the scale of the number 1500 would be
1.5E2
Both the width of the field and number of significant digits are required. A possible exponential phrase might be:
E10.4
When this phrase is applied to 1500 the result (between the vertical bars) would be
| 1.500E2|
Text can be displayed with a type A phrase. How to do this depends on the data. Simple arrays require a format field for each character in a line while nested arrays are displayed in one field.
To display a simple character array use the repeat feature:
NAMES←4 6 ⍴ 'NUTS SCREWSBOLTS NAILS '
COSTS← 0.05 0.03 0.20 0.01
QUANT← 150 200 4 1000
(Reprinted from Gilman L, Rose AJ. APL: An Interactive Approach, ed. 2
rev. Wiley, 1976, ch 21, p. 195)
'6A1,F10.2,I5,F10.2' fmt NAMES,COSTS,QUANT,[1.1]COSTS×QUANT
NUTS 0.05 150 7.50
SCREWS 0.03 200 6.00
BOLTS 0.20 4 0.80
NAILS 0.01 1000 10.00
Note that the number of times the type A field is repeated is equal to the width of the NAMES array.
To display a nested character array, use the field width:
NAMES← 'NUTS' 'SCREWS' 'BOLTS' 'NAILS'
'A7,F10.2,I5,F10.2' fmt NAMES,COSTS,QUANT,[1.1]COSTS×QUANT
NUTS 0.05 150 7.50
SCREWS 0.03 200 6.00
BOLTS 0.20 4 0.80
NAILS 0.01 1000 10.00
Text that should appear within a numeric field is called a decorator. Such text should be bracketed by delimiters. There are four sets
'2M⊂(⊃N<)>CF14.2' fmt 13599 ¯13399
13,599.00 (13,399.00)
This is a workspace to create html files. html is a text markup scheme used by the world wide web. At its most basic level html is a collection of tags, that is a word enclosed in angle brackets, which instruct a web browser how to display the text.
The html workspace is an implementation of the xml workspace. html creates a set of functions that return marked up text for inclusion in an html document. It creates a function for each html tag that takes as its optional, left argument a lexicon of attributes (see workspace lex)and as its right argument the text to be marked up.
A Hello World html document might be coded like this:
head←html∆head html∆title 'Hello World'
html∆html head , html∆body html∆h1 'Hello World'
<html><head><title>Hello World</title></head>
<body><h1>Hello World</h1></body></html>
Import is an apl workspace to import arrays from text files.
Import∆file reads a file from disk and returns an array of rank 2, that is, rows and columns. It will determine whether the file is tab or comma delimited and will determine which columns contain numeric data and covert those strings to numbers.
Import∆table examines the array returned by import∆file. It will remove blank columns, heading rows at the begining of the file and footer rows at the end. It will also replace blank cells in numeric columns with zeros.
Lex is an implementation of a name-value store for apl. Functions here allow one to create such a store, add name and retrieve a value for a name.
Functions are:
Returns an empty store
Returns a new lexicon with the supplied name–value inserted into the supplied lexicon.
Returns the value of the supplied name.
Predicate to return true if the supplied item is in fact a lexicon
Returns the list of names in the supplied lexicon
Returns a list of values in the supplied lexicon
We use and-dot-equals to do a sequential search of the list of names. For other hashing algorithms try lex1.
Lex1 is an implementation of a name-value store for apl using a hash. Functions here allow one to create such a store, add name and retrieve a value for a name.
Functions are
Returns an empty store.
Returns a hash with the supplied name–value inserted into the supplied lexicon.
Returns the value of the supplied name.
Predicate to return true if the supplied item is in fact a lexicon.
Returns the list of names in the supplied lexicon.
Returns a list of values in the supplied lexicon.
Returns a bucket number mapped to the supplied key.
Returns the number of items in each bucket.
We use ⎕io+prime|+/⎕ucs key to compute the hash. It has
two features:
This workspace supports name–value pairs in a component file. It uses the APLComponentFiles written by Blake McBride and distributed with GNU APL.
The workspaces uses the same API as lex.apl. That is the following functions:
Returns a list of component names.
Creates a component file ant returns a file handle.
Assigns a component to a name. Will append or overwrite as appropriate.
Returns the component with the supplied name.
Determines if the supplied file handle (an integer) is in fact a lexicon based component file.
Returns all of the components of the file up to a maximum of cl∆∆max.
Opens a database. This is a wrapper for CF_DBCONNECT. The left argument identifies the type of database while the right (db_spec) varies by that type. Postgress wants a connection string of ’host=hostname user=username password=password dbname=data_base_name’ while sqlite wants a file name.
Opens a component file and returns a file handle. cl∆open_db must be called before cl∆open as the component files are stored in an SQL database.
Close a component file.
Closes the connection to the database. Function is a wrapper for CF_DBDISCONNECT.
Workspace to print directly from APL.
Function to print plain text. On success lpr returns 0, otherwise an error code greater than 0. See the man page for lpr.
Printer is a lexicon of various printing parameters. See lpr∆USLetter and lpr∆a4 below.
Function to assemble a printer lexicon for US Letter paper (8.5 inch by 11 inch). Right argument printer is the name (as CUPS understands it) of your target printer.
See your system administrator for this name.
If you are the system adiministrator and are in the dark try ‘man cups‘.
Function to Assemble a printer lexicon for A4 paper. Try ‘1 lpr∆din ’A4’ for this size of A4 paper. Talk to your system administrator for the printer name.
Margins for these printer lexicons were selected to yeild printouts that I liked. If you don’t, roll your own. A printer lexicon is made up of the following items:
We are printing fixed width text after all. For something fancier:
Function to render and then print html. While one must supply a printer lexicon, the only item that is used here is the name of the printer.
Function returns 0 on success and something greater on failure. See the man page for lpr.
This function relies on html2ps written by Jan Kärrman; Dept. of Information Technology; Uppsala University; Sweden and is Free and Open Source Software. It is a package generally available from Ubuntu and Debian.
Workspace to generate passwords.
Function returns a pseudo-randomly generated password from the given character set. If length is omitted, eight characters will be generated.
Returns a list of alphabetic characters.
Returns the ten digits in Arabic numerals.
Returns a set of characters that are not alpha-numeric. Wikipedia does not agree with what a dingbat is. This function can’t read Wikipedia.
It also can’t read the instructions you have from some rogue website. Type them in. viz:
pw∆password pw∆alpha,pw∆numeric,'!*%^$'
preferences is a workspace that displays the currently active settings in the gnu-apl preference file. The command ’pref∆current pref∆find_prefs’ does the displaying.
pref∆current pref∆find_prefs Welcome No Color No SharedVars Enabled (default) LIBREF-3 /usr/local/lib/apl/wslib3 LIBREF-4 /usr/local/lib/apl/wslib4 LIBREF-5 /usr/local/lib/apl/wslib5 READLINE_HISTORY_LEN 500 READLINE_HISTORY_PATH .apl.history NABLA-TO-HISTORY Modified (default) BACKUP_BEFORE_SAVE yes CONTROL-Ds-TO-EXIT 0 Profile 2 disable_Quad_GTK yes # disable ⎕cwGTK disable_Quad_FIO no # do not disable ⎕FIO disable_native_functions yes # disable A ⎕FX B (native fnctions) disable_Quad_PLOT yes # disable ⎕PLOT disable_Quad_SQL yes # disable ⎕SQL disable_SAVE_command yes # disable )SAVE and )DUMP ws CIN-SEQUENCE 1b 5b 30 3b 33 32 3b 34 30 6d // ESC [0;32;40m COUT-SEQUENCE 1b 5b 30 3b 33 37 3b 34 30 6d // ESC [0;37;40m CERR-SEQUENCE 1b 5b 30 3b 33 31 3b 34 30 6d // ESC [0;31;40m UERR-SEQUENCE 1b 5b 30 3b 33 31 3b 34 30 6d // ESC [0;31;40m RESET-SEQUENCE 1b 5b 30 3b 33 37 3b 34 30 6d // ESC [0;37;48m CLEAR-EOL-SEQUENCE // CLEAR-EOS-SEQUENCE // Keyboard NOCURSES (default) LIBREF-0 /home/dalyw/workspaces LIBREF-1 /home/dalyw/wslib1 LIBREF-2 /home/dalyw/wslib2 LIBREF-6 /home/dalyw/wslib6 LIBREF-7 /dilworth/workspaces LIBREF-8 /mnt/workspaces LIBREF-9 /mnt/wslib1
Workspace to seek answers from your users.
Function prints the query and returns the user’s answer. If one of the four keywords quit, top, done, or back are entered, prompt will return an integer. 1 for quit, 2 for top, 3 from done, and 4 for back when ⎕io is set to 1.
This should allow the programmer to react:
[1] ans←prompt 'Last Name ... ' [2] →(utl∆numberp ans)/keyword [3] ⍝ Deal with actual data .. [n]→0 [n+1] →(top,quit,done,back)[ans] [n+2] top:
I use ’top’ to return the user to the first prompt in the program; ’quit’ to exit the program making no changes; ’done’ to exit the program making changes; and ’back’ to return the users to the previous prompt.
Workspace to provide the next entry in a sequence. Each sequence is a prefix and a count.
Function returns the next item in a sequence with prefix key, ’Invoice0001, Invoice0002 ... for example.
Function sets the value of a given sequence. Some do not like to start at zero. Use seq∆set.
'foo' seq∆set 10001
t1←(⊂seq∆next 'foo'),⊂seq∆next 'foo'
t1
foo10001 f0010002
Function returns the next integer in the sequence
Function erase existing sequences.
A lexicon of existing sequences with each prefix as a key and its value as the last number generated.
Workspace to implements a simple stack using functional programming constructs.
This means that whenever the stack changes the function returns a modified stack. Accordingly four functions are required:
Function returns a new, empty stack.
Function returns stack old with item as the first item on old.
Function returns the first item on stack old.
Function returns stack old with the first item removed.
To use this library, first initiate the stack:
my_stack←st∆init
Add items to the stack:
my_stack←my_stack st∆push item
Pop items from the stack:
item←st∆peek my_stack my_stack← st∆pop my_stack
The stat workspace provides functions to perform statistical calculation and to organize data for statistical analysis. It is very much a work in process.
Current Functions are:
Returns a count of the items in the vector.
Computes the y intercept of the Least Mean Squares function given the slope the that line.
Computes the slope of a least mean square regression. In the data (a level 2 array) data[;1] is the dependent data and data[;2] the independent.
Calculates the coefficient of correlation of the regression by use of stat∆lms∆slope and stat∆lms∆intercept.
Returns the range of a data set. The is the highest amount less the lowest.
Returns the population variance.
Returns the population standard deviation.
Returns the sample mean of a vector.
Returns the sample variance.
Returns the sample standard deviation.
Returns the median.
Workspace provides access to utf8 files.
Function to write text to a new file or replace an existing file.
Function return 0 on success.
Function returns the text of file_name.
File returns a list of the files in directory path.
Functions pipes text to an operating system command.
Functions returns the standard out of a command.
Utl is a collection of generally usefull routines.
Display help about a function. This routine prints the function header and any comments that immediately follow.
Tests whether item is a number. Returns true or false. See also utl∆numberis.
Tests whether an item can become a number. IE is utl∆numberp ⍎ test true?
Tests whether item is a character vector.
Returns a left justified array of characters with the minimum number of trailing spaces. At least one line of the array will have no trailing spaces.
Converts all white space to spaces and then removes duplicate spaces.
web_log parses web server logs in the common format. Web servers keep a log of requests made of the server and the servers response. Analyzing those logs may confirm the relative popularity of a site monitoring those logs may yield insight into what content changes were well received.
The process of parsing a log is load the log into the current workspace with function log∆loadFile, apply several tests to the file and remove irrelevant entries, and count the number of entries.
To that end web_log provides a function, log∆filter_hits, to remove entries that I thought were not actual visits to a site. Each line of this function applies a test to each line of the log using one of two operators, log∆filter and log∆not_filter. To use log∆filter_hits call it with a left argument of your ip address and a right argument of the log returned by log∆loadFile. viz.:
access←log∆loadFile '/dilworth/www.dalywebandedit.com/logs/access_log.2022-06-30'
⍴access
217
⍴'68.81.178.238' log∆filter_hits access
53
log∆filter_hits is a good example of how to use this workspace. It applies a series of tests to the log data, removing lines as it does its work.
∇log∆filter_hits[⎕]∇
∇
[0] new←ip log∆filter_hits old
[1] ⍝ Function returns the number of page requests
[2] new←200 log∆test_status log∆filter old
[3] new←'bot' 'spider' 'google' 'crawler' log∆test_agent_contents log∆not_filter new
[4] new←'js' 'css' 'jpg' 'gif' 'ico' 'png' log∆test_suffix log∆not_filter new
[5] new←ip log∆test_ip log∆not_filter new
[6] new←'/wordpress1/wp-cron.php' log∆test_file log∆not_filter new
∇
log∆filter and log∆not_filter are operators that apply a test to each line of the log. log∆filter returns lines where the test is true while log∆not_filter returns lines where the test is false.
web_log provides a series of tests. Each test’s name starts with ’log∆test_’. Provided tests are:
Each log line contains an agent field. This test is true when the supplied string can be found in the agent field. In log∆filter_hits we remove requests from spiders, crawlers and bots by looking for these name in the agent fields
Each log line contains a target field, or what the client requested. This test is true if fname is that target. The test ignores the CGI query that might be included in the target.
Each log line has a referrer which if present is the URL of the web page that contained the link the client clicked on. The test is true if the host matches the host in the URL.
The log line includes the ip address of the client and this test is true when the line’s ip address is in the set of ip addresses supplied.
This test returns true if the referrer field is blank.
The line’s status field contains an integer, defined by the Hypertext Transfer Protocol (HTTP). A code of 200, OK, means this server supplied a page. Wikipedia has a complete list of codes.
Function tests if the file name from the target field has a suffix in the list of supplied suffixes.
There are ten fields in each log line. The web_log workspace provides a function for each field.
Function returns the client’s ip address.
Function returns the client’s user name. This field is populated only when the server requires login using HTTP.
Function returns the time stamp in the log line. The time stamp is generally in coordinated universal time. The next field, time zone is the adjustment to local time.
The workspace also provides log∆ParsedTime which returns a vector of five integers: year month day hour second.
The adjustment needed to apply to UTC for local time.
The HTTP request. An HTTP request is a three part datum. The first part is the HTTP method, the second the requested URL, and the third the version of HTTP.
HTTP defines ’POST’ ’GET’ among others as methods.
web_log provides the functions log∆method and log∆target for the first two parts.
Function returns the HTTP status code that the server used to respond to this request. The status code is an integer defined by HTTP. 200, OK and 404, NOT FOUND are the most usual.
Function returns the count of bytes returned by the server to the client.
This function returns the URL of the page the client clicked on to make the current request.
The agent field contains information about the program on the client’s computer that made the request.
This is a workspace for printing arrays (on paper). It offers control over many aspects of the printed page.
The workspace generates an HTML page which can then be loaded into one’s browser and printed.
Reports are a complex data structure initiated and maintained through functions in the wp workspace.
A report consists of an array, report heading, and stylesheet. For each cell in the array additional attributes can be saved for formatting the report cell by cell.
The basic process is:
Use the function wp∆init to create a report. Init requires a short character string as its sole argument and will print that string in the report as an identifier for the report.
The report itself is a lexicon of Id, Data, Attributes, Stylesheet, Head_ln01 through Head_ln03, and Author. wp∆init will use the environment’s USER for the Author.
By default, a report’s data is a single cell of 0.
wp requires an array of rank 2 and expects that descriptions of the rows and columns will be included in the array. There is no enforced limit on the size of the array. However, the target browser that displays the array may limit what you can put on the screen.
The HTML coding is designed to allow the browser to determine column widths, text wrapping within a cell and other features. As the browser window size is changed the browser should reformat the report.
Users experienced in cascading style sheets can add style sheet classes to establish actual widths, heights and placements for a more fixed presentation Stylesheet.
An attribute is a name-value pair. Each cell can have a list of attributes. Two are in general use ’format’ and ’class’. Other attributes will be passed to your browser as attributes of the cell. For example a cell has three attributes, format, class and passed. the coded HTML would be:
<td class=''number'' passed=''user-defined''>', CF14.2 fmt cell_amount, '</td>...</tr>'
The format attribute is used by the workspace to format the number of the cell. APL2’s format by example (’55,550.00’ ⍕ number) and function fmt fmt, a partial implementation of STSC’s ⎕FMT can be used.
The class attribute selects the style sheet class to use for the cell Stylesheet.
Accessor functions for attributes have a prefix wp∆attr∆. They are:
Function returns a lexicon (Lex, a name-value store) of attributes.
Function adds, or replaces the name value pair. The function expects a lexicon of name value pairs. For example:
rpt←rpt wp∆attr∆setCellAttr 15 3,⊂lex∆from_alist 'format' 'CF10.2' 'class' 'number'
Function returns a portion of the Attribute array.
Funtion sets a rectangular portion of the Attribute array. If one lexicon is given that lexicon will be copied to each cell in the rectangle. If more than one lexicon is given, the size and shape of the final argument must match the size and shape of the rectangle.
The style sheet is a lexicon of classes that describe how to present a cell in the data. Each class within the sheet is a lexicon of properties. An example style sheet is returned by the function wp∆defaultSS.
Properties are defined in the global variable wp∆ssp∆propertyList which is derived from the W3.com cascading style sheet standards. The standards define what values each property may take.
Function returns an empty style sheet.
Function returns a style sheet with classes defined for a typical array.
Function returns a new style sheet class.
Function adds a property to a class. Properties are name-value pairs. For instance:
r4←wp∆ssr∆init '.page-head' r4←r4 wp∆ssr∆assignProp 'font-size' 'large'
Assign a class to a stylesheet. Function will silently replace an existing class.
ss←ss wp∆ss∆assignClass r4
Parts of the final report require their own class.
All of these classes are included in the default style sheet. You may not like the results if they are omitted from the published style sheet.
Many properties are looking for a distance. Distances can be fixed, relative or percentages (a number followed by %). There should be no white space between the amount the the measure.
Fixed distances can be measured in:
Relative units are determined by the font:
You may be thinking that because elements on web pages are measured in numbers of pixels, and because pixels are considered a relative measurement, that they are the answer to all font size problems. It would be nice if the were. For some designers, control over text at the pixel level is intoxicating.
– _Jennifer Niederst Robbins, Web Design in a Nutshell, O’Reilly_
This workspace provides functions to implement an xml schema and a cascading stylesheet to display that schema.
Each element in a document using this schema can be generated from a function of that name.
Each element-function will use its left argument as the element’s attributes and its right as the element’s content. Attributes are stored in lexicons (see workspace lex).
For example, assume an HTML5 schema has been implemented:
attr←(lex∆init) lex∆assign ’class’ ’right-justified’ tag←attr html∆p ’Now is the winter of our discontent’ tag <p class="right-justified">Now is the winter of our discontent" </p>‘
The schema is implemented by calling xml∆MkTagFns or xml∆MkClosedTagFns:
Creates a function for elements named tag
Creates a function for a empty tag (<br\>).
We’ve put an example application in html_test.apl. This workspaces first provides functions for a subset of HTML5 and then defines html∆fmt_table to take an array of rank two and return an HTML page.
Browsers use cascading stylesheets display an xml documnet. They consist of a lexicon (see workspace lex) of selectors and rules. Each rule consists of a lexicon of properties and values.
There are two functions for cascading stylesheets:
Function returns a nested lexcion from the text of a stylesheet.
Function returns the text of a stylesheet from a nested lexicon.
One may integrate the workspaces in APL Library with APL Package Manger by David B. Lamkins. Most workspaces in the library can be imported into the package manager as a package. The script ancillary/install_packages.apl will do the heavy lifting.
Packages are first apl workspaces that use the same prefix in the name of all workspace symbols, second are utf8 files that one may )COPY into one’s workspaces and third, stateless.
The package manger must be installed before calling ancillary/install_packages.apl. Download the manager at https://github.com/TieDyedDevil/apl-pkg and then execute script install.sh.
Test the installation by )load pkg; ]pkg packages. Read the manager’s tutorial with ]pkg read apl-packager 2.
Install the library’s workspaces by executing the script ancillary/install_packages.apl.
Confirm the installation by )load pkg and ]pkg packages.
Test the installation by )load pkg, ]pkg load unit_test, and test∆menu test∆menu∆items
Copyright © 2000, 2001, 2002, 2007, 2008 Free Software Foundation, Inc. https://fsf.org/ Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.
The purpose of this License is to make a manual, textbook, or other functional and useful document free in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with or without modifying it, either commercially or noncommercially. Secondarily, this License preserves for the author and publisher a way to get credit for their work, while not being considered responsible for modifications made by others.
This License is a kind of “copyleft”, which means that derivative works of the document must themselves be free in the same sense. It complements the GNU General Public License, which is a copyleft license designed for free software.
We have designed this License in order to use it for manuals for free software, because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software does. But this License is not limited to software manuals; it can be used for any textual work, regardless of subject matter or whether it is published as a printed book. We recommend this License principally for works whose purpose is instruction or reference.
This License applies to any manual or other work, in any medium, that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. Such a notice grants a world-wide, royalty-free license, unlimited in duration, to use that work under the conditions stated herein. The “Document”, below, refers to any such manual or work. Any member of the public is a licensee, and is addressed as “you”. You accept the license if you copy, modify or distribute the work in a way requiring permission under copyright law.
A “Modified Version” of the Document means any work containing the Document or a portion of it, either copied verbatim, or with modifications and/or translated into another language.
A “Secondary Section” is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document’s overall subject (or to related matters) and contains nothing that could fall directly within that overall subject. (Thus, if the Document is in part a textbook of mathematics, a Secondary Section may not explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regarding them.
The “Invariant Sections” are certain Secondary Sections whose titles are designated, as being those of Invariant Sections, in the notice that says that the Document is released under this License. If a section does not fit the above definition of Secondary then it is not allowed to be designated as Invariant. The Document may contain zero Invariant Sections. If the Document does not identify any Invariant Sections then there are none.
The “Cover Texts” are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that the Document is released under this License. A Front-Cover Text may be at most 5 words, and a Back-Cover Text may be at most 25 words.
A “Transparent” copy of the Document means a machine-readable copy, represented in a format whose specification is available to the general public, that is suitable for revising the document straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing editor, and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters. A copy made in an otherwise Transparent file format whose markup, or absence of markup, has been arranged to thwart or discourage subsequent modification by readers is not Transparent. An image format is not Transparent if used for any substantial amount of text. A copy that is not “Transparent” is called “Opaque”.
Examples of suitable formats for Transparent copies include plain ASCII without markup, Texinfo input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard-conforming simple HTML, PostScript or PDF designed for human modification. Examples of transparent image formats include PNG, XCF and JPG. Opaque formats include proprietary formats that can be read and edited only by proprietary word processors, SGML or XML for which the DTD and/or processing tools are not generally available, and the machine-generated HTML, PostScript or PDF produced by some word processors for output purposes only.
The “Title Page” means, for a printed book, the title page itself, plus such following pages as are needed to hold, legibly, the material this License requires to appear in the title page. For works in formats which do not have any title page as such, “Title Page” means the text near the most prominent appearance of the work’s title, preceding the beginning of the body of the text.
The “publisher” means any person or entity that distributes copies of the Document to the public.
A section “Entitled XYZ” means a named subunit of the Document whose title either is precisely XYZ or contains XYZ in parentheses following text that translates XYZ in another language. (Here XYZ stands for a specific section name mentioned below, such as “Acknowledgements”, “Dedications”, “Endorsements”, or “History”.) To “Preserve the Title” of such a section when you modify the Document means that it remains a section “Entitled XYZ” according to this definition.
The Document may include Warranty Disclaimers next to the notice which states that this License applies to the Document. These Warranty Disclaimers are considered to be included by reference in this License, but only as regards disclaiming warranties: any other implication that these Warranty Disclaimers may have is void and has no effect on the meaning of this License.
You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3.
You may also lend copies, under the same conditions stated above, and you may publicly display copies.
If you publish printed copies (or copies in media that commonly have printed covers) of the Document, numbering more than 100, and the Document’s license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in addition. Copying with changes limited to the covers, as long as they preserve the title of the Document and satisfy these conditions, can be treated as verbatim copying in other respects.
If the required texts for either cover are too voluminous to fit legibly, you should put the first ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto adjacent pages.
If you publish or distribute Opaque copies of the Document numbering more than 100, you must either include a machine-readable Transparent copy along with each Opaque copy, or state in or with each Opaque copy a computer-network location from which the general network-using public has access to download using public-standard network protocols a complete Transparent copy of the Document, free of added material. If you use the latter option, you must take reasonably prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public.
It is requested, but not required, that you contact the authors of the Document well before redistributing any large number of copies, to give them a chance to provide you with an updated version of the Document.
You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you release the Modified Version under precisely this License, with the Modified Version filling the role of the Document, thus licensing distribution and modification of the Modified Version to whoever possesses a copy of it. In addition, you must do these things in the Modified Version:
If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the list of Invariant Sections in the Modified Version’s license notice. These titles must be distinct from any other section titles.
You may add a section Entitled “Endorsements”, provided it contains nothing but endorsements of your Modified Version by various parties—for example, statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard.
You may add a passage of up to five words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by you or by arrangement made by the same entity you are acting on behalf of, you may not add another; but you may replace the old one, on explicit permission from the previous publisher that added the old one.
The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version.
You may combine the Document with other documents released under this License, under the terms defined in section 4 above for modified versions, provided that you include in the combination all of the Invariant Sections of all of the original documents, unmodified, and list them all as Invariant Sections of your combined work in its license notice, and that you preserve all their Warranty Disclaimers.
The combined work need only contain one copy of this License, and multiple identical Invariant Sections may be replaced with a single copy. If there are multiple Invariant Sections with the same name but different contents, make the title of each such section unique by adding at the end of it, in parentheses, the name of the original author or publisher of that section if known, or else a unique number. Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work.
In the combination, you must combine any sections Entitled “History” in the various original documents, forming one section Entitled “History”; likewise combine any sections Entitled “Acknowledgements”, and any sections Entitled “Dedications”. You must delete all sections Entitled “Endorsements.”
You may make a collection consisting of the Document and other documents released under this License, and replace the individual copies of this License in the various documents with a single copy that is included in the collection, provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects.
You may extract a single document from such a collection, and distribute it individually under this License, provided you insert a copy of this License into the extracted document, and follow this License in all other respects regarding verbatim copying of that document.
A compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a storage or distribution medium, is called an “aggregate” if the copyright resulting from the compilation is not used to limit the legal rights of the compilation’s users beyond what the individual works permit. When the Document is included in an aggregate, this License does not apply to the other works in the aggregate which are not themselves derivative works of the Document.
If the Cover Text requirement of section 3 is applicable to these copies of the Document, then if the Document is less than one half of the entire aggregate, the Document’s Cover Texts may be placed on covers that bracket the Document within the aggregate, or the electronic equivalent of covers if the Document is in electronic form. Otherwise they must appear on printed covers that bracket the whole aggregate.
Translation is considered a kind of modification, so you may distribute translations of the Document under the terms of section 4. Replacing Invariant Sections with translations requires special permission from their copyright holders, but you may include translations of some or all Invariant Sections in addition to the original versions of these Invariant Sections. You may include a translation of this License, and all the license notices in the Document, and any Warranty Disclaimers, provided that you also include the original English version of this License and the original versions of those notices and disclaimers. In case of a disagreement between the translation and the original version of this License or a notice or disclaimer, the original version will prevail.
If a section in the Document is Entitled “Acknowledgements”, “Dedications”, or “History”, the requirement (section 4) to Preserve its Title (section 1) will typically require changing the actual title.
You may not copy, modify, sublicense, or distribute the Document except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense, or distribute it is void, and will automatically terminate your rights under this License.
However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and finally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is reinstated permanently if the copyright holder notifies you of the violation by some reasonable means, this is the first time you have received notice of violation of this License (for any work) from that copyright holder, and you cure the violation prior to 30 days after your receipt of the notice.
Termination of your rights under this section does not terminate the licenses of parties who have received copies or rights from you under this License. If your rights have been terminated and not permanently reinstated, receipt of a copy of some or all of the same material does not give you any rights to use it.
The Free Software Foundation may publish new, revised versions of the GNU Free Documentation License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. See https://www.gnu.org/licenses/.
Each version of the License is given a distinguishing version number. If the Document specifies that a particular numbered version of this License “or any later version” applies to it, you have the option of following the terms and conditions either of that specified version or of any later version that has been published (not as a draft) by the Free Software Foundation. If the Document does not specify a version number of this License, you may choose any version ever published (not as a draft) by the Free Software Foundation. If the Document specifies that a proxy can decide which future versions of this License can be used, that proxy’s public statement of acceptance of a version permanently authorizes you to choose that version for the Document.
“Massive Multiauthor Collaboration Site” (or “MMC Site”) means any World Wide Web server that publishes copyrightable works and also provides prominent facilities for anybody to edit those works. A public wiki that anybody can edit is an example of such a server. A “Massive Multiauthor Collaboration” (or “MMC”) contained in the site means any set of copyrightable works thus published on the MMC site.
“CC-BY-SA” means the Creative Commons Attribution-Share Alike 3.0 license published by Creative Commons Corporation, a not-for-profit corporation with a principal place of business in San Francisco, California, as well as future copyleft versions of that license published by that same organization.
“Incorporate” means to publish or republish a Document, in whole or in part, as part of another Document.
An MMC is “eligible for relicensing” if it is licensed under this License, and if all works that were first published under this License somewhere other than this MMC, and subsequently incorporated in whole or in part into the MMC, (1) had no cover texts or invariant sections, and (2) were thus incorporated prior to November 1, 2008.
The operator of an MMC Site may republish an MMC contained in the site under CC-BY-SA on the same site at any time before August 1, 2009, provided the MMC is eligible for relicensing.
To use this License in a document you have written, include a copy of the License in the document and put the following copyright and license notices just after the title page:
Copyright (C) year your name. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled ``GNU Free Documentation License''.
If you have Invariant Sections, Front-Cover Texts and Back-Cover Texts, replace the “with…Texts.” line with this:
with the Invariant Sections being list their titles, with
the Front-Cover Texts being list, and with the Back-Cover Texts
being list.
If you have Invariant Sections without Cover Texts, or some other combination of the three, merge those two alternatives to suit the situation.
If your document contains nontrivial examples of program code, we recommend releasing these examples in parallel under your choice of free software license, such as the GNU General Public License, to permit their use in free software.
| Jump to: | ⎕
A C D E F H I L N P U |
|---|
| Jump to: | ⎕
A C D E F H I L N P U |
|---|