Landmark Graphics Corporation World Wide Technology Conference Houston, Texas December 1, 1994 A Tcl/Tk and Expect Tutorial by: Will Morse BHP Petroleum Copyright 1994 Will Morse. Permission is given to freely copy and distribute this paper as long as there is no charge except real and actual mechanical copying costs and as long as this notice is kept with each copy so others may copy it as well. The opinions expressed are the author's own and do not necessarily reflect the opinions or policies of The Broken Hill Proprietary Company, Limited, or its various divisions. Your milage may vary. INTRODUCTION: Shell scripts are the frequent tool of Landmark systems administrators. Whether Bourne shell, C-shell, or Korn shell, shell script programs have distinct limits. * They usually run in an Xterm window, and require the user to have some ancillary knowledge of Unix commands. * They do not take advantage, or take only the most trivial advantage, of the mouse. * They do not have the advantages of scroll bars, radio buttons and so on, so they must ask questions and take user input, even if the default is acceptable. For some programs, this can be twenty or more questions, even if the answer is just a tap of the enter key. * It is unusual to have these programs "hanging" on the Command Menu (launcher). In addition, shell script programs cannot have a useful two-way dialog with a program such as bcm2d or zap. If you want to run ten bcm2d jobs in a shell script, you still have to watch the screen and every now and then respond to questions. Most systems can not run several zap jobs at once. It would be nice to be able to "stack" them in a shell script and run them sequentially over night. Tcl/Tk and expect act on the level of shell scripts, but provide the capability of drawing X-windows "widgets" and of operating programs without a human constantly watching. As a systems administrator or data administrator, these programs can save you time and increase your productivity because you can make many simple, and even complex, operations available to the geologist / geophysicist user directly. You can also save time by being able to run many data loading and other processing operations unattended overnight. All these programs are FREE. The most you might pay is $60 or so for a CD-ROM containing the programs which you can then install on your system. There are also several consulting firms in the Houston area (see page 19) that will do all the installation and give you training, for a fee, of course. In this tutorial, we will discuss: Definitions page 3 Seismic / SeisWorks applications page 4 A Simple Expect Program page 6 A Simple Tcl/Tk (wish) Program page 8 Basic Tcl page 10 Variables page 10 Set page 10 Embedded Execution page 10 Arithmetic page 11 If-then-Else page 11 Loops page 12 Files page 12 Procedures page 13 Regular Expressions page 13 A Simple Expectk Program page 15 Manuals and Documentation page 17 Where to Get It page 18 Where to Get Support page 19 DEFINITIONS: Tcl pronounced "tickle", is a fairly simple and straight forward computer language similar to a shell language. Tcl stands for Tool Command Language. Tk is a set of libraries that can be used by a string oriented computer language such as Tcl to draw X-windows "widgets" such as list boxes, menus, and so on. Tk is most often used with Tcl, but will also work with other languages such as perl and scheme. expect is an extended version of Tcl which can serve as a "robot operator" to run certain kinds of programs, in particular programs like Landmark's bcm2d and bcm3d and zap. expectk is a version of expect that uses the X-windows drawing capability of Tk. wish is a combined interpreter (shell) for Tcl and Tk. wish stands for "windowing shell". widget a component of an X-windows window. Examples of widgets are: scroll-bars, select lists, push buttons, radio buttons, pull-down and pop-up menus. A special kind of widget is the "canvas", which is used for diagrams, graphs, and other drawings. Tcl, while it has many merits, is generally not of interest to this audience except in combination with one of the other programs. Tcl/Tk is dramatically easier than C, C++, and the Motif and X libraries. Still, it is not the easiest place to start of this group. We will cover some of the basic ideas in this paper that will get you to where you can read and understand the manuals. expect is the easiest program in the group. You can write useful scripts in expect without understanding Tcl at all. A little Tcl will help you expand these scripts into powerful tools. expectk is not hard, but combines elements from the other programs so it is best to learn at least a little about Tcl, Tk, and expect before getting into expectk. SEISMIC AND SEISWORKS APPLICATIONS: Applications written in these languages that are related to seismic interpretation and to SeisWorks include: * run.bcm2d and run.bcm3d These programs are used to build a shell script that can run bcm jobs at night with no operator attention. * License signup sheet. This on-line signup sheet lets users sign up for licenses at specified times. Users can plan their usage without having to go to a central paper signup sheet. With the "Check licenses" feature, a person who has signed up for a license can find out if someone else is "borrowing" some of their time. * Eclipse Control Panel The Eclipse Reservoir Simulator requires a number of routine questions be answered each time it runs. The Eclipse Control Panel presents these questions as select boxes, radio buttons, and text entry boxes. The default answers are pre-selected. If you want to run the defaults, just click okay. * ZAP-later. This program collects the parameters needed to run a ZAP job and allows that job to be run overnight at a specified time. The user does not have to be logged in to the system at the time the ZAP is run. * Programs like plist, odacheck, and other SeisWorks utilities are easily run in a windows environment with results displayed in list boxes. * CGM Plotting Control Panel This program lets the user select paper or transparency media, up to 5 copies, rotated display, remove after plotting or keep. There is a feature to help find the CGM file. A previewer (incorporating gplot, a FREE program) lets the user see the general look of the plot before commiting to paper. * Planimeter A simple planimeter program that puts the digitized points in a file the way you want. * Replacement for launcher. This program works like the standard LGC launcher except it lets you specify seperator lines between menus, lets you use icons for selections instead of just text, allows selections to be turned on and off. * GUI's for CWP / SU CWP / SU (Center for Wave Phenomon, Seismic Unix) is a set of free seismic processing programs. These programs are similar to shell script programs in the way they operate. Tcl/Tk can provide a Graphic User Interface to these programs and make them much easier to use. A SIMPLE EXPECT PROGRAM: Expect is basically a "robot operator" that executes a program for you. When this program puts a message out, thinking it is going to you as the program user, expect intercepts that message and gives the program the appropriate reply. run.bcm2d 1 #! /usr/local/bin/expect -- 2 set timeout -1 3 spawn bcm2d [lindex $argv 0] 4 expect *number :* 5 exec sleep 1 6 send "r\r" 7 expect "*Ready, or A to Abort :*" 8 exec sleep 1 9 send "r\r" 10 expect eof 11 exit You can put several instances of this program in a basic shell script and run the program overnight: #! /bin/sh run.bcm2d file01.pcl run.bcm2d file02.pcl run.bcm2d file03.pcl run.bcm2d file04.pcl ... run.bcm2d file17.pcl run.bcm2d file18.pcl run.bcm2d file19.pcl run.bcm2d file20.pcl Line 1 is the "shebang" line. It tells Unix what shell to use to run this program. Line 2 turns off the timeout feature. This feature is useful in programs that dial into remote databases and so on, but not for bcm2d. The program runs the bcm2d program using spawn (3). The part [lindex $argv 0] takes the "zeroth" item from the list $argv and uses that as the file bcm2d processes. lindex stands for "list index", and, like Unix programs everywhere, 0 is the first item (and in this case, the only item) in the list. Every time the program stops to ask what reel number you want to use (4), expect supplies an answer "r" (6). The * in front and after "number :" tell expect that the question will contain these characters, but that there will be other characters as well. The \r (6) is the return key you would have typed. There is a 1 second "sleep" (5) between the time the question is asked by bcm2d and the time expect gives the answer because bcm2d is expecting a human to answer and isn't quite ready to listen so quickly after asking. Every time bcm2d asks if you are ready (7), expect fills in the "r" (9), once again a sleep (8) with \r for the return key. The eof (10) is the end of file. When bcm2d (or any other program) finishes and exits, it closes its standard output file. expect detects this. If you did not wait for eof, expect would just quit after answering the Ready prompt (9), and take down bcm2d with it. A SIMPLE TCL/TK (WISH) PROGRAM: This simple program executes the Landmark "plist" command and puts the results in a scrollable listbox. You could easily "hang" this command on the OpenWorks Command Menu (launcher). This example has unusual spacing to make it easier to show the various parts of the program, there is no spacing requirement in Tcl/Tk. The italicized numbers are for the explanation below, they do not exist in the file. 1 #! /usr/local/bin/wish -f 2 wm title . {Landmark Project List} 3 wm geometry . +250+250 4 wm minsize . 1 1 5 label .label -text {Landmark Project List} \ 6 -font 10x20 7 frame .f -borderwidth 5 8 scrollbar .f.y -command {.f.list yview} 9 scrollbar .f.x -orient horiz \ 10 -command {.f.list xview} 11 listbox .f.list -relief sunken \ 12 -xscroll {.f.x set} \ 13 -yscroll {.f.y set} 14 pack append .f \ 15 .f.y {right filly} \ 16 .f.x {bottom fillx} \ 17 .f.list {top fillx filly expand} 18 button .ok -text {OK} \ 19 -command {destroy .} 20 pack append . \ 21 .label {top fillx} \ 22 .f {top fillx} \ 23 .ok {top fillx} 24 set fh [open "|plist" r] 25 while {! [eof $fh]} { 26 gets $fh line 27 .f.list insert end $line 28 } Line 1 is the "shebang" line to tell Unix which interpreter to use. Line 2 uses the wm command to pass the title to Motif to display on the title bar. Line 3 uses the wm command to pass the location on the screen to Motif. If you do not use this, you will have to click the mouse button to "realize" the window. Line 4 gives the minimum dimensions of the window. This turns on the ability to change the size of the window using the Motif side handles. Line 5 Defines a label on the window giving the text and the font. Note the escaped return. You cannot just start on the next line. Line 7 creates a frame. A frame is a assembledge of smaller pieces that will be treated as one piece. Line 8 describes the vertical scrollbar. The way this works is that the scroll bar computes a position, them tells .f.list, the listbox, to move to that position. Line 9 is the same for the horizontal scrollbar Line 11 describes the listbox and associates it with the scrollbars. Line 14 "packages" the components of the scroll box, that is, the listbox and the two scrollbars. Line 18 creates a "button". When this button is "pressed", it executes the command. In this case, the command is to destroy the base widget (named .) which exits the program. Line 20 packages the base widget with the label, the frame, and the button. They are all set to "top" so they will appear in order from the top. Line 24 is a Tcl command (see page 12). It opens a pipe in which it executes the "plist" command. It sets the filehandle for the result in fh. Line 25 is a Tcl command, it tests the end-of-file status for the piped plist command. As long as there are more lines of output from plist, the program executes the target commands. Line 26 is a Tcl command. gets reads a line from the plist command and puts it in the variable named "line". Line 27 inserts the line into the listbox at the end. Line 28 closes the while loop. The program creates the window and then waits for user to do something. If the something is moving the scroll bars, the program executes those commands. If the something is clicking the OK button, the program exits. BASIC TCL: Tcl has hundreds of features. This is a barely adequate overview to help us understand the other programs. Variables: A Tcl variable is any set of letters and numbers, but it is best for the first character to be a letter. All variables in Tcl are character strings, they only become numbers during arithmetic operations. Variables can be in arrays. env is the array of environmental variables. set home $env(HOME) Set: The set command give a variable a value. set pi 3.1415926 set name "World Wide Technology Conference" Set without a value reports the value that the variable already contains: set pi returns 3.1415926. A shorthand way to report a value (and by far the most frequent way to do this, is to precede the variable name with a $: set morePi $pi Embedded Execution: Tcl has an embedded execution using square brackets that is basically analogous to the back quote execution in Bourne shell except that embedded execution can be nested in Tcl. set a [pwd] set a [sort $list] set a [sort [list $x $y $z]] Arithmetic: The great bulk of arithmetic in Tcl takes place in either the conditions of an if-then-else or in an expr command. You can not have a command: set a $a + 1 It needs to be: set a [expr $a + 1] There are some commands which act as though they had an expr command in front of them: set a [sin $angle] set a [expr sin($angle)] but not set a [atan2 $y $x] this must be set a [expr atan2($y,$x)] If-then-else: The Tcl if-then-else is pretty straight forward: if {condition} then {action} else {action} This is frequently written: if {$a == $b} then { puts "true" } else { puts "false" } You must escape the return before the then and the else to write: if {$a < $b} \ { puts "true" } \ else \ { puts "false" } This is because Tcl does not have a specific line termination character (such as the semicolon in perl). For and While: These are also pretty straight forward: while {condition} {action} for {start} {test} {increment} {action} For example: while {! [eof $file]} { gets $filehandle line } for {set i 1} {i < 10} {incr i 1} { set a [expr 10 * $i] puts "$a" } Files: Reading and writing are normally done using the gets and puts commands. It is not necessary to open STDIN, STDOUT, or STDERR. Other files must be opened explicitly. Files are closed implicitly when the program terminates, but you can explicitly close them if you want. File commands include: open note, open needs to be "set xx [open...] close filehandle eof filehandle gets filehandle variable puts filehandle "string with $variables" For Example: set fhIn [open "/etc/passwd" r] set fhOut [open "/home/hacker/passwd" w] while {! [eof $fhIn]} { gets $fhIn input puts $fhOut "$input" } close $fhIn close $fhOut The file can be a pipe to another program. This is signified by placing the pipe symbol at the BEGINNING of the command name: set fhIn [open "|cat /etc/passwd" r] set fhOut [open "|wc -l" w] Procedures and Subroutines: Procedures are defined with the proc statement: proc hypot {x y} { set h [sqrt $x*$x+$y*$y] return $h} To call a procedure, write the procedure as a statement: gets x gets y set hy [hypot $x $y] puts "Hypot = $hy" You can give a procedure default values: proc maybe {{window .ww}} { toplevel $ww button $ww.b -text ... } You can execute a Unix command with the exec command: set longevity [exec uptime] BOTH STANDARD OUTPUT AND STANDARD ERROR will be set into the variable (in this case, longevity). Regular Expressions: No Unix program is complete without at least one regular expression. The regexp command handles regular expressions. This can be in an if statement, such as: if [rexexp {^[0-9][0-9]*} $number] then {...} It can also be used to extract values. regexp { *([0-9][0-9]*), *([0-9][0-9]*), *(..*)} \ match xLocation yLocation buttonPressed This can be greatly simplified using variables: set D "[0-9][0-9]*" # one or more digits set ANY "..*" # one or more characters regexp " *($D), *($D), *($ANY)" match X Y button You could not define the spaces as set WS " *" and regexp "$WS($D),$WS($D),$WS($ANY)" match X Y button because Tcl would be confused whether $WS($D) was referring to two variables next to each other, or an array variable $WS indexed by $D. A SIMPLE EXPECTK PROGRAM: This program performs a simple action and puts up a "Click to Continue" button to let you know that it finished. 1 #! /usr/local/bin/expectk -- 2 button .ok -text {Running Tar} -command exit 3 pack .ok -fill x -fill y 4 exec tar -cvf /dev/rst2 /p?/project1 5 expect eof 6 .ok config -text {Click to Continue} You will want to have the Tk description in place (lines 2 and 3) before running the command (line 4) or expectk will place a rather annoying default box on the screen until it gets directions on how to fill it. Line 6 is used to change the text on the button. A slightly more useful version of this program would be: 1 #! /usr/local/bin/expectk -- 2 wm geometry . +250+250 3 wm minsize . 1 1 4 button .ok -text {Running The Command} \ 5 -borderwidth 10 \ 6 -relief sunken \ 7 -state disabled \ 8 -disabledforeground black \ 9 -activebackground grey \ 10 -activeforeground green \ 11 -fg black \ 12 -bg pink \ 13 -font 10x20 \ 14 -command exit 15 pack .ok -fill x -fill y 16 set xx [exec $argv ] 17 expect eof 18 .ok config -state normal \ 19 -relief raised \ 20 -bg green \ 21 -text {Push to Continue} This program will run an arbitrary command supplied as an argurement and tell you when it is done. The program is called by: flag.run "ls /p?" The double quotes make the ls and the /p? into one argurment, otherwise the program would only execute the ls. The program uses a lot of colors basically to demonstrate that they can be used. It is not necessary to use so many colors. Lines 2 and 3 set up the window so that it is only the necessary size. The default size is rather larger. Line 4 gives the initial text. This text will be changed by line 21. Lines 5, 6, and 7 set up characteristics of the button. The borderwidth is used to give the amount of relief the button will have. The state is disabled so that the user cannot click the button before the program is finished. Once the prgoram is finished, the state is changed to normal in line 18. The relief is sunken. The relief is changed to raised in line 19. Lines 8 - 12 give colors for various states the button can be in. Note that there is not an option for disabledbackground. active means that the mouse pointer is positioned over the button. The button cannot become active if it is disabled. Line 16 is where the command is executed. The set xx [] is used to capture the output in a string in the program so that is is not displayed on the xterm. The config command in line 19 is used to change parameters specified originally in line 4. MANUALS AND DOCUMENTATION: Each of these programs has a "man page" that is installed with the program. This has no additional cost, but has the typical "reminder for people who already know" style of man pages generally. The definitive reference for Tcl and Tk is: Tcl and the Tk Toolkit John Ousterhout Published by Addison-Wesley ISBN: 0-201-63337-X The definitive reference for expect and expectk, and a good resource for Tcl, is: Exploring Expect by Don Libes Published by O'Reilly & Associates, ISBN: 1-56592-090-2 These books are available at BookStop in Houston and at other fine bookstores. The Energy Related Unix User's Group Unix Cookbook contains contributed Tcl/Tk scripts of interest to the petroleum exploration and production technical user. WHERE TO GET IT: If you have an Internet connection, all these programs can be obtained by anonymous ftp from ftp.cme.nist.gov Other sources are CD-ROM's. Some CD-ROM's are distributed with books. Two good places to look for these programs are: Prime Time Freeware for Unix, $60.00 ISBN 1-881957-04-7 Unix Power Tools, $59.95 ISBN 0-679-79073-X These books are available at BookStop in Houston and at other fine bookstores. WHERE TO GET SUPPORT: The best support for all these languages is in the Internet Newsgroup: comp.lang.tcl There are at least two companies in Houston that will help with Tcl and Tk on a consulting basis. Neosoft Paranet There are some people in the Energy Related Unix User's Group (ERUUG) in Houston that are using Tcl/Tk and Expect. To join ERUUG, call Analisa Smith at (713) 621-0022. ======================================================================= This is both papers and the end of this message. From jgerber@solar.rtd.utk.edu Mon Dec 19 13:36:09 1994 Received: from solar.rtd.utk.edu (SOLAR.RTD.UTK.EDU [128.169.112.24]) by Starbase.NeoSoft.COM (8.6.9/8.6.9) with SMTP id NAA27470 for; Mon, 19 Dec 1994 13:36:08 -0600 Message-Id: <199412191936.NAA27470@Starbase.NeoSoft.COM> X-Provider: NeoSoft, Inc.: Internet Service Provider (713) 684-5969 Received: by solar.rtd.utk.edu; Mon, 19 Dec 94 14:35:23 EST From: jgerber@solar.rtd.utk.edu (John J. Gerber) Sender: jgerber@solar.rtd.utk.edu Subject: Papers on Perl and Tcl/Tk To: will@starbase.neosoft.com Date: Mon, 19 Dec 1994 14:35:23 -0500 (EST) X-Mailer: ELM [version 2.4 PL22] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 714 Status: OR I'm always a little hesitant sending email to an address other then the original poster. James Huang (james@oslonett.no) posted to comp.lang.perl a message stating that you had presented two award winning papers at the Landmark Graphics World Wide Technology Forum. The papers were about Perl and Tcl/Tk, and anyone wishing to lay hands on copies of these papers could send email requesting these papers. I would very much be interested in these papers. Thank you in advance for any assistance you can give. -- John -- InterNet: jgerber@solar.rtd.utk.edu VoiceNet: (615) 974-2909 StampNet: Office of Research Services FaxNet: (615) 974-6508 211 Hoskins Library Knoxville, TN 37996