From: [email]j.p.h@comcast.net[/email] (Joe Halpin)
Newsgroups: comp.unix.shell
Subject: comp.unix.shell FAQ - Answers to Frequently Asked Questions
Summary: This posting answers questions which come up with some frequency on comp.unix.shell. It should be read by anyone with a question about shell programming before posting questions to the newsgroup.
Followup-To: comp.unix.shell
Archive-name: unix-faq/shell/sh
Posting-Frequency: monthly
Version: $Id: cus-faq.html,v 1.11 2005/09/01 17:39:36 jhalpin Exp $
Maintainer: Joe Halpin
This FAQ list contains the answers to some Frequently Asked Questions often seen in comp.unix.shell. It spells "unix" in lower case letters to avoid arguments about whether or not Linux, FreeBSD, etc are unix. That's not the point of this FAQ, and I'm ignoring the issue.
This document as a whole is Copyright (c) 2003 Joe Halpin. It may be copied freely. Exceptions are noted in individual answers.
Suggestions, complaints, et al, should be sent to the maintainer at [email]j.p.h@comcast.net[/email] or posted to comp.unix.shell
There are two levels of questions about shells.
One is the use of the shell itself as an interface to the operating system. For example, "how do I run a program in the background, and go on with other things?". Or "how do I setup environmental variables when I log in?".
The other level is how to write shell scripts. This often involves having the shell execute unix utilities to perform part of the work the shell script needs to accomplish, and requires knowledge of these utilities, which isn't nominally in the scope of shell programming. However, unless the question involves something other than standard unix utilities, it should be included in this FAQ.
Standard unix utilities are defined by either POSIX or the Single Unix Specification. These are now joined and are normally abbreviated as "POSIX/SUS". This specification can be found at
http://www.opengroup.org/onlinepubs/007904975/toc.htm
The man pages found on that web page define standard behavior for any given utility (or the shell itself). However, you should also check the man page on your system for any utility or shell you need to use. There isn't always a perfect correspondence between the standard and a particular implementation (in fact, I'm not sure there's any case in which they perfectly correspond).
There is also an Austin Group FAQ, which describes the standardization effort in more detail ls at
http://www.opengroup.org/austin/faq.html
Other good web sites that provide information about shells and shell programming (including OS utilities) include:
http://www.shelldorado.com/
http://cfaj.freeshell.org/shell/
http://www.faqs.org/faqs/by-newsgroup/comp/comp.unix.shell.html
This FAQ is available at
http://home.comcast.net/~j.p.h/
http://www.newsville.com/cgi-bin/getfaq?file=comp.unix.shell/comp.unix.shell_FAQ_-_Answers_to_Frequently_Asked_Questions
---------------------------
The predictable legal stuff
---------------------------
The answers given in this FAQ list are provided with the best intentions, but they may not be accurate for any particular
shell/os. They may be completely wrong for any shell/os. If you don't test the answers, that's a bug in your procedures.
There are no guarantees for the answers or recommendations given in this document. In fact, I don't even claim to have tested any or all of them myself. Many of the answers here have been contributed by one or more regular participants in the newsgroup, who I believe to be competent (certainly more competent than I am), but THERE ARE NO GUARANTEES.
Did I really need to make that all uppercase? Hopefully not, but there are a lot of lawyers around with too much time on their hands, so I want to make it clear that THERE ARE NO GUARANTEES about the accuracy of answers in this FAQ list. This is, hopefully, an aid to people trying to learn shell programming, but it is NOT a supported product. You have to figure out for yourself whether or not the answers here work for what you're trying to do.
Under no circumstances will the maintainer of this FAQ list, or any contributors to it, be held liable for any mistakes in this
document. If the answers work for you, well and good. If not, please tell me and I'll modify them appropriately so that this will be more useful.
If you don't agree to that, don't read any farther than this. Reading beyond this point indicates your agreement.
If you do test the answers and find a problem, please send email to the maintainer (see above), so it can be corrected, or (preferably) post a question to the newsgroup so it can be discussed and corrected if there's a problem.
A number of people have contributed to this FAQ, knowingly or unknowingly. Some of the answers were taken from previous postings in the group, and other people contributed questions and answers directly to the maintainer, which you are welcome to do as well.
Among the contributors is Heiner Steven, who also provided the momentum to get this FAQ list started. He maintains a web site about shell programming that has a lot of good stuff in it.
http://www.shelldorado.com/
======================================================================
CONTENTS:
[url=http://bbs.chinaunix.net/viewthread.php?tid=551026&page=1#pid3647960][color=Green]0 COPYING
0a. Glossary
Google
POSIX/SUS ("the standard")
UUOC
dotfile
portable
race condition
shebang
shells
top-posting[/color]
[url=http://bbs.chinaunix.net/viewthread.php?tid=551026&page=1#pid3647961][color=Orange]0b. Notes about using echo[/color]
[url=http://bbs.chinaunix.net/viewthread.php?tid=551026&page=1#pid3647964][color=Green]1. How can I send e-mails with attached files?
2. How can I generate random numbers in shell scripts?
3. How can I automatically transfer files using FTP with error checking?[/color]
[url=http://bbs.chinaunix.net/viewthread.php?tid=551026&page=1#pid3647967][color=Orange]4. How can I remove whitespace characters within file names?
5. How can I automate a telnet session?
6. How do I do date arithmetic?[/color]
[url=http://bbs.chinaunix.net/viewthread.php?tid=551026&page=1#pid3647969][color=Green]7. Why did someone tell me to RTFM?
8. How do I create a lock file?
9. How can I convert DOS text files to unix, and vice versa?
10. How can a shell prompt be set up to change the title of xterm?[/color]
[url=http://bbs.chinaunix.net/viewthread.php?tid=551026&page=1#pid3647972][color=Orange]11. How do I get the exit code of cmd1 in cmd1|cmd2
12. Why do I get "script.sh: not found"
13. Why doesn't echo do what I want?[/color]
[url=http://bbs.chinaunix.net/viewthread.php?tid=551026&page=1#pid3647974][color=Green]14. How do I loop through files with spaces in their name?
15. how do I change my login shell?
16. When should I use a shell instead of perl/python/ruby/tcl...
17. Why shouldn't I use csh?[/color]
[url=http://bbs.chinaunix.net/viewthread.php?tid=551026&page=1#pid3647975][color=Orange]18. How do I reverse a file?
19. How do I remove last n lines?
20. how do I get file size, or file modification time?[/color]
[url=http://bbs.chinaunix.net/viewthread.php?tid=551026&page=1#pid3647976][[color=Green]21. How do I get a process id given a process name? Or, how do I find out if a process is still running, given a process ID?
22. How do I get a script to update my current environment?
23. how do I rename *.foo to *.bar?[/color]
[url=http://bbs.chinaunix.net/viewthread.php?tid=551026&page=2#pid3647979][color=Orange]24. How do I use shell variables in awk scripts
25. How do I input the user with a timeout?[/color]
[url=http://bbs.chinaunix.net/viewthread.php?tid=551026&page=2#pid3647981][[color=Green]26. How do I get one character input from the user?
27. why isn't my .profile read?
28. why do I get "[5" not found in "[$1 -eq 2]"?
29. How do I exactly display the content of $var (with a \n appended).
30. How do I split a pathname into the directory and file?[/color]
[url=http://bbs.chinaunix.net/viewthread.php?tid=551026&page=2#pid3647982][color=Orange]31. How do I make an alias take an argument?
32. How do I deal with a file whose name begins with a weird character
33. Why do I lose the value of global variables that are set in a loop
34. How do I batch an FTP download/upload?[/color]
[url=http://bbs.chinaunix.net/viewthread.php?tid=551026&page=2#pid3647985]Appendix A: Some example scripts
Appendix B: References. These
correspond with numbers in square brackets (e.g. [1]) which may appear
in the text.
======================================================================
[ 本帖最后由 r2007 于 2005-12-3 17:41 编辑 ]
r2007 回复于：2005-05-25 21:41:14
ANSWERS
0 COPYING
Some contributors may copyright their submissions and license them
differently than this document.
[1] Chris F.A. Johnson. Examples marked with COPYING[1] were
contributed by Chris F.A. Johnson. He has copyrighted these
examples, and licensed them under the GNU General Public
License (GPL). Copying them directly into another script will
cause that script to also come under the GPL. For details see
http://www.fsf.org/licenses/licenses.html
0a.Glossary
-------------------------------
Google
Google is one of the search engines on the Internet. It took
over dejanews some years ago, and now is the standard reference
when directing someone to a past thread one some topic. This is
a very good place to start when researching a question about
shell programming (and just about anything else).
http://groups.google.com/advanced_group_search
-------------------------------
POSIX/SUS ("the standard")
POSIX (Portable Operating System Interface) and SUS (Single Unix
Specification) have been joined into one standard. This is what
people usually mean when they refer to "the standard" in
discussions about unix. When people in this group refer to the
POSIX shell, they are talking about the shell prescribed by this
specification. You can find this standard at
http://www.opengroup.org/onlinepubs/007904975/toc.htm
-------------------------------
UUOC
This is short for "Useless use of cat". It's used to point out
that some example script has used cat when it could have used
redirection instead. It's more efficient to redirect input than
it is to spawn a process to run cat. For example
$ cat file | tr -d 'xyz'
runs two processes, one for cat and one for tr. This is less
efficient than
$ tr -d 'xyz' < file
In general, "cat file | somecommand" can be more efficiently
replaced by "somecommand < file"
or (especially for multi-file input)
$ somecommand file [file ...]
but check the man page for "somecommand" to find out if it will
accept this syntax.
For more details about this, as well as other things like it, see
http://rhols66.adsl.netsonic.fi/era/unix/award.html
-------------------------------
dotfile
This refers to a file which starts with '.' (a dot). These files
are not shown in directory listings without the -a (or -A in
newer versions of ls - check the man page on your system) option
to ls. Often they are configuration files, subdirectories used
by applications to store configuration files, NFS swap files, et
al.
-------------------------------
portable
The word "portable" means different things to different people,
in different situations, which is to say, there isn't one
definition of "portable".
At one extreme, a portable script is one which will work under
any shell, on any operating system. At this end of the spectrum,
there is no such thing as a portable shell script (some
operating systems don't even have shells). If we confine the
operating system to unix (which would make sense since this is
comp.unix.shell), the only truly portable scripts are those
which make no use of built-in shell facilities or syntax, but
which only call external utilities. For example
echo Hello World
would probably qualify. However, that doesn't do anyone much
good.
Given that there are probably few (if any) scripts which have to
meet such a standard, a more frequent use of the word "portable"
indicates the degree to which a script will run under different
shells and/or different operating environments.
For example, if you're writing an installation script for an
application, and the platforms on which that application runs
are defined, then the problem is pretty well bounded. The choice
of shell is one which is available on all required platforms,
and the syntax to be used is the smallest subset of all the
variants of that shell on the target platforms.
The degree to which your shell script needs to be portable has
to be determined by you, or the requirements you've been given
for the script.
-------------------------------
race condition
This is a situation in which two entities (processes, threads,
etc) are trying to access a shared resource, or perform the same
action, and the result depends on the order of execution of the
two entities.
-------------------------------
shebang
This is the first line of a shell script, which indicates to the
operating system which interpreter (shell) it should invoke to
interpret the script. It has the form
#!/path/to/shell [ argument ]
where /path/to/shell might be /bin/sh, /usr/local/bin/bash, etc.
This line is only interpreted by the operating system. That is,
if a shell script (test.sh) is executable and run from the
command line by typing its name and giving the script as an
argument, as in
$ sh test.sh
then sh interprets test.sh. For sh, the shebang line is simply a
comment, and is ignored.
-------------------------------
shells
A number of shells are discussed in this group, including
sh
csh
pdksh
ksh88
ksh93
tcsh
zsh
rc
es
bash
ash
dash
These (and more) are names of shells which are referenced in the
group. A comparison of some of these is available at
http://www.faqs.org/faqs/unix-faq/shell/shell-differences/
However, it does not make specific the differences between
ksh88, ksh93 and pdksh, which are not entirely compatible.
-------------------------------
top-posting
A. top posting
Q. What's the most irritating way to respond on usenet?
Please see the following:
http://catb.org/~esr/jargon/html/T/top-post.html
http://www.uwasa.fi/~ts/http/quote.html
http://members.fortunecity.com/nnqweb/
http://www.guckes.net/mail/editing.html
======================================================================
[ 本帖最后由 r2007 于 2005-12-3 16:19 编辑 ]
r2007 回复于：2005-05-25 21:41:55
0b. Notes about using echo
This isn't really a FAQ, but discussions about using echo come up
often enough that it seems reasonable to have something about it in
the FAQ list.
The echo command is not consistent in the handling of its arguments
from implementation to implementation. Sometimes a string with
backslash quoted characters will be interpreted in one way, and
sometimes another.
Also, if the string being echoed wasn't built into the script
itself, then it could have shell metacharacters in it, which could
confuse things. In cases where external input is used to build a
string to be echoed the string typically should be quoted.
For example
s="a string with\na newline and\ta tab"
Following are some results with various shells:
------
bash:
$ echo "$s"
a string with\na newline and\ta tab
$ echo -e "$s"
a string with
a newline and a tab
-------
pdksh:
$ echo "$s"
a string with
a newline and a tab
$ echo -e "$s"
$ echo "$s"
a string with
a newline and a tab
--------
ksh88:
$ echo "$s"
a string with
a newline and a tab
$ echo -e "$s"
-e a string with
a newline and a tab
-------
ksh93:
$ echo "$s"
a string with\na newline and\ta tab
$ echo -e "$s"
-e a string with\na newline and\ta tab
Note that ksh93 makes the handling of arguments system dependent
when they contain '\', and/or the first argument begins with '-'.
http://www.cs.princeton.edu/~jlk/kornshell/doc/man93.html
POSIX does not allow the -e option. It also makes the result of
using -n or any string with '\' in it implementation-defined.
However, on XSI-conforming systems, it disallows options, and
defines the use of backslash-quoted characters.
In general, the behavior of echo is system and/or shell dependent
if its arguments contain a backslash, or its first argument is -n
or -e.
The biggest problem with echo is when using it to output strings
that the script got externally (e.g. user input, or reading from a
file). These strings may have '\' characters in them for
example. In this case, results may not be what you expect.
print is available in some shells, although printf(1) is perhaps
more portable. Additionally, a here document will give predictable
results in that it will not expand escape sequences.
cat <<EOF
$s
EOF
produces
a string with\na newline and\ta tab
So consider not using echo unless you are sure what will happen,
given the shell you're using.
======================================================================
[ 本帖最后由 r2007 于 2005-12-3 16:27 编辑 ]
r2007 回复于：2005-05-25 21:42:36
1. How can I send e-mails with attached files?
a. Use uuencode
This is the simplest way to do this. For example
$ uuencode surfing.jpeg surfing.jpeg | mail [email]someone@some.wher[/email]e
To send regular text as well
$ (cat mailtext; uuencode surfing.jpeg surfing.jpeg) |
mail [email]someone@some.wher[/email]e
b. Use MIME
$ metasend -b -t [email]someone@some.wher[/email]e -s "Hear our son!" \
-m audio/basic -f crying.au
These examples are adapted from
http://www.shelldorado.com/articles/mailattachments.html which
goes into much more detail about this.
c. Use pine (with a patch) or mutt
======================================================================
2. How can I generate random numbers in shell scripts?
This depends on the shell, and the facilities available from the
OS.
a. Some shells have a variable called RANDOM, which evaluates to a
different value every time you dereference it. If your shell has
this variable,
$ number=$RANDOM will produce a random number.
b. Some systems have a /dev/urandom device, which generates a
stream of bits. This can be accessed using the dd(1) utility. An
example of this (from a more extensive discussion of different
techniques at http://www.shelldorado.com/scripts/cmds/rand)
n=`dd if=/dev/urandom bs=1 count=4 2>/dev/null | od -t u4 | \
awk 'NR==1 {print $2}'`
also:
od -vAn -N4 -tu4 < /dev/urandom
c. Use a utility such as awk(1), which has random number generation
included. This approach is the most portable between shells and
operating systems.
awk 'BEGIN {srand();print rand()}'
Note that this doesn't work with older versions of awk. This
requires a version supporting the POSIX spec for srand(). For
example, on Solaris this will not work with /usr/bin/awk, but
will with nawk or /usr/xpg4/bin/awk.
Also, if you call this line more than once within the same
second, you'll get the same number you did the previous time.
======================================================================
3. How can I automatically transfer files using FTP with error
checking?
First, there are tools to do that: curl, wget, lftp, ncftp. But,
they are generally not part of the base system (you need to
install them).
zsh (version 4 and above) provides a FTP facility, see "info -f
zsh -n 'zsh/zftp Module'"
#! /usr/bin/zsh
zftp open host user passwd || exit
zftp get /remote/file > /local/file; r=$?
zftp close && exit r
With your system "ftp" command, two ways:
1- using "ftp -n". Without the -n option, ftp expects user
interaction to enter the password, so you'd need to use
"expect". With "-n", you provide the user and password as any
other FTP command.
#! /bin/sh
ftp -n << EOF
open ftp.domain.org
user anonymous ${LOGNAME:-`who am i`}@
binary
get /remote/file /local/file
bye
EOF
The error checking can't be made correctly (if "open" fails, the
"user" command will be still sent even if it shouldn't).
2- using ~/.netrc
If you put:
<<
machine ftp.domain.org
login mylogin
password mypasswd
macdef init
binary
get /remote/file /local/file
bye
>>
(with the trailing empty line) in your ~/.netrc (ensure it's not
world readable) and then run "ftp ftp.domain.org", ftp will find
the matching "machine" entry in your ~/.netrc and use the
parameters provided there to make the ftp transaction.
Those work at least on Linux, FreeBSD, Solaris, HPUX
======================================================================
[ 本帖最后由 r2007 于 2005-12-3 16:30 编辑 ]
r2007 回复于：2005-05-25 21:43:27
4. How can I remove whitespace characters within file names?
File names in unix can contain all kinds of whitespace characters,
not just spaces. The following examples only work with spaces,
adjust accordingly.
a. Use the substitution capabilities of awk, sed, et al.
f=`printf '%s\n' "$filename" | sed 's/ /_/g'`
f=`printf '%s\n' "$filename" | awk '{gsub(" ","_");print $0}'`
f=`printf '%s\n' "$filename" | tr ' ' _`
Add characters to the tr command line as needed (see the man
page for tr to find out the available escape sequences).
Additionally (although not exactly a one-liner)
f=`tr ' ' _ <<EOF
$filename
EOF
`
See section 0a "Notes about using echo" for why echo is not used
here.
b. Use the substitution capabilities of the shell if it has
them. Check the man page for your shell (probably under a
section named something like "Parameter expansion") to see. For
example:
f=${filename// /_}
With zsh:
autoload -U zmv
zmv '* *' '$f:gs/ /_/'
It should be noted that the zmv solution renames the files (call
mv internally and adress several problems that may arise) while
the other solutions only update a variable (and then, renaming
the files may involve a quite complicated script to do it
reliably).
======================================================================
5. How can I automate a telnet session?
This is outside the realm of shell programing, per se. You need
a more special purpose scripting language such as expect. See
http://expect.nist.gov/
Perl scripts can also do this with the Telnet module from CPAN.
======================================================================
6. How do I do date arithmetic?
This depends on exactly what you have in mind.
a. Finding yesterday's date
The GNU version of date has some nice features in this
respect. For example
To find yesterday's date
$ date --date yesterday
To find tomorrow's date
$ date --date tomorrow
See the man page for GNU date for other options. It can also
provide dates more than one day in the past/future.
The FreeBSD version of date also provides extensions that can
do things like this.
$ date
Wed Oct 22 13:48:29 CDT 2003
$ date -v-1d
Tue Oct 21 13:45:16 CDT 2003
Playing with the TZ variable isn't a reliable method. If you
need to do something like this, but don't have GNU or FreeBSD
date available, see section g. "Arbitrary date arithmetic".
b. Determining relative ages of files
If you want to determine whether or not one file is older than
another, you can (with bash, pdksh, ksh93) do
$ [[ file1 -ot file2 ]] && echo file1 is older
or you can use find to search a directory tree for files that
are newer/older than some file:
$ find . -name '*.c' -newer test.c
c. Finding elapsed time
If you want to find elapsed time, perhaps because you want to
know when some operation has timed out, some shells (bash, ksh
zsh [,??]) have a SECONDS variable which tell how many seconds
have elapsed since the invocation of the shell, or since the
last time it was set.
ksh93 has a floating point SECONDS which is locale dependent.
In zsh 4.1 and above one can be made floating point with: float
SECONDS
zsh 4.1 and above also has $EPOCHSECONDS for seconds since
1970-1-1 0:0:0 UTC (see zsh/datetime module).
d. Determining leap year
A leap year in the Gregorian calendar is defined as a year which
is evenly divisible by 4, however, if it's also evenly divisible
by 100 then it's not a leap year unless it's also evenly
divisible by 400. It gets worse than that, actually, but this
is as far as I go :-).
In the Julain calendar which was used before in Europe, only the
years divisible by 4 where leap years.
The standard "cal" utility performed the switch between Julian
and Gregorian calendar in september 1752 (see cal 9 1752) which
corresponds to the date used in England. The Gregorian calendar
(created by Pope Gregory III) was first used in 1582 in many
other countries.
One possibility for a ksh function to do this (after 1600 AD/CE)
is
isleap()
{
y=$1
four=$(( $y % 4 ))
hundred=$(( $y % 100 ))
fourhundred=$(( $y % 400 ))
if [ $four -eq 0 ];then
if [ $hundred -eq 0 ];then
if [ $fourhundred -eq 0 ];then
echo leap year
else
echo not a leap year
fi
else
echo leap year
fi
else
echo not a leap year
fi
}
Or, valid with any date with the same calendar switch day as
POSIX cal's (POSIX syntax):
is_leap_year() # args: year
# NB: year before year 1 is year -1, not 0.
{
[ "$1" -lt 0 ] && set -- "$(($1 + 1))"
[ "$(($1 % 4))" -eq 0 ] && {
[ "$(($1 % 100))" -ne 0 ] || [ "$(($1 % 400))" -eq 0 ] \
|| [ "$1" -le 1752 ]
}
}
Or in any Bourne shell (see COPYING[1]):
is_leap_year() { ## USAGE: is_leap_year [year]
isl_year=${1:-`date +%Y`}
case $isl_year in
*0[48] |\
*[2468][048] |\
*[13579][26] |\
*[13579][26]0|\
*[2468][048]00 |\
*[13579][26]00 ) _IS_LEAP_YEAR=1
return 0 ;;
*) _IS_LEAP_YEAR=0
return 1 ;;
esac
}
On FreeBSD, use the -f option to date(1) to pass in the
(supposed) February 29 in the current year and then print it the
day of the month again to see if there really is such a date
(note that you need -j as well as -f, otherwise date(1) thinks
you want to set the clock):
if [ $(date -jf%Y%m%d $(date +%Y0229) +%d) = 29 ]; then
echo Leap year!
fi
e. Determining the last day of a month.
There are a number of possibilities for doing this which have
been mentioned in the group. The following is a sampling:
In any Bourne-type shell (in conjunction with is_leap_year() as
given above, when month is February) (see COPYING[1]):
days_in_month() { ## USAGE: days_in_month [month [year]]
if [ -n "$1" ]
then
dim_m=$1
dim_y=$2
else
eval `date "+dim_m=%m dim_y=%Y"`
fi
case $dim_m in
9|09|4|04|6|06|11) _DAYS_IN_MONTH=30 ;;
1|01|3|03|5|05|7|07|8|08|10|12) _DAYS_IN_MONTH=31 ;;
2|02) is_leap_year ${dim_y:-`date +%Y`} &&
_DAYS_IN_MONTH=29 || _DAYS_IN_MONTH=28 ;;
esac
[ ${SILENT_FUNCS:-0} -eq 1 ] || echo $_DAYS_IN_MONTH
}
With GNU date:
year=2003
month=9
date -d "$year/$month/1 +1 month -1 day" +%d
With FreeBSD date use the -v-1d option to date(1) to get the day
before the first day of the next month:
$ MONTH=12
$ date -v-1d -jf%Y-%m-%d $(date +%Y-$(((MONTH+1)%12))-01) +%d
31
In the shell using cal (But beware of implementations of cal
which print more than one month at a time):
month=9 ; year=2003 # adjust
##
for lday in `cal $month $year` ; do : ; done
echo $lday
## or
set -- `cal $month $year` ; eval lday=\${$#}
echo $lday
In ksh, bash and zsh:
: $(cal)
days_in_month=$_
In zsh:
days_in_month=${$(cal)[-1]}
f. Determining the day of the week for a given date.
This algorithm is known as Zeller's congruence. An explanation
of it is available from the Dictionary of Algorithms and Data
Structures web page at NIST:
http://www.nist.gov/dads/
Also, a fuller explanation is available at
http://www.merlyn.demon.co.uk/zeller-c.htm#ZC
An example in C, with a short explanation, is given at
http://wwwcdf.pd.infn.it/MLO/Calendars/Notes.html#zeller
A shell (ksh93) implementation of a homework assignment (given
for illustration only - don't turn this in as yours - you might
be sorry if it's wrong :-)
dayofweek()
{
# Implementation of a homework assignment given at
# http://carbon.cudenver.edu/~traup/fa02/lec/hw3.html
#
# call with day: 1 - 31
# month: March = 1, Jan & Feb are months 11 and
# 12 of the previous year.
# year: The year of the century
# c: The previous century
#
# For example, for July 4, 1989,
# m = 5, d = 4, y = 89, and c = 19,
# while for January 25, 1989,
# m = 11, d = 25, y = 88, and c = 19.
#
# The output is the day of the week with Sunday = 0,
# Monday = 1, etc.
d=$1
m=$2
y=$3
c=$4
A=$(( ($m * 13 - 1) / 5 ))
B=$(( $y / 4 ))
C=$(( $c / 4 ))
D=$(( $A + $B + $C + $d + $y - ($c * 2) ))
echo $(( $D % 7 ))
}
On FreeBSD, use the -f option to date(1) to pass in the date of
interest and +%A to print the day of the week:
$ date -jf%Y-%m-%d 2000-01-01 +%A
Saturday
(Use +%u or +%w if you want the weekday as a number. See the
strftime(3) manpage for details.)
g. Arbitrary date arithmetic
To do arbitrary date calculations is more complicated. One
possibility is to call an external utility, or a program in
another scripting language, which has this built in. For
example, perl has wrappers for the unix time functions built in,
so it can provide some relief in this regard. C programs can
also be easily written to do date arithmetic (see the examples
section). One thing to keep in mind, however, is that unix time
functions are, strictly speaking, limited to the range of time
between January 1 1970 at midnight, and January 19, 2038 at
3:14:07. C/Perl programs which calculate dates outside this
range might work, or they might not, that would depend on the
implementation.
To do arbitrary date arithmetic in the shell itself is also
possible. An article provided on the web by SysAdmin magazine
describes one way to do this.
http://www.samag.com/documents/s=8284/sam0307b/0307b.htm
Another possibility is given in the examples section, from
http://groups.google.com/groups?selm=n6d6zalnpk.fsf%40ogion.it.jyu.fi
See also
http://groups.google.com/groups?selm=slrnbvnhu2.3v.stephane.chazelas@spam.is.invalid
On FreeBSD, the -f and -v options to date(1) cover most things
you might want to do, with the caveat that only dates within the
range mentioned above are defined. Dates outside that range are
not guaranteed to work.
Also, zsh 4.1 and above has the zsh/datetime module that
provides the $EPOCHSECONDS and the strftime function.
h. Getting the number of seconds since the epoch
- GNU date has the %s format option which returns the epoch
time.
- More portably, use awk.
awk 'BEGIN {srand(); printf("%d\n", srand())}'
This works because srand() sets its seed value with the
current epoch time if not given an argument. It also returns
the previous seed value, so the second call gives the epoch
time.
Note that this doesn't work with older versions of awk. This
requires a version supporting the POSIX spec for srand(). For
example, on Solaris this will not work with /usr/bin/awk, but
will with nawk or /usr/xpg4/bin/awk.
Depending on scheduling, when the call is actually executed,
etc, this might be off by a second.
- Another way is to use perl if you have it.
perl -le 'print time'
- Also, zsh 4.1 and above has the zsh/datetime module that
provides the $EPOCHSECONDS and the strftime function.
======================================================================
[ 本帖最后由 r2007 于 2005-12-3 17:44 编辑 ]
r2007 回复于：2005-05-25 21:43:59
7. Why did someone tell me to RTFM?
Because you didn't :-)
RTFM is part of Usenet lingo, and means "Read The F-ing Manual".
Generally people say this when someone asks a question that is
asked so often, and is answered plainly in some relevant man page,
that they're tired of seeing it asked.
http://catb.org/~esr/jargon/html/R/RTFM.html
So RTFM, and the FAQs first before asking. Also, if you're new to
the group, search Google Groups
http://groups.google.com/advanced_group_search
before asking questions. And please don't post your homework
questions to the group unless you've tried to figure them out, and
have some specific questions. People will generally be happy to
help you with your homework if you post what you've got and ask
specific questions.
======================================================================
8. How do I create a lock file?
Very carefully :-)
The scheduler can stop one process in the middle of a non-atomic
operation, and run another one, which wants to perform the same
operation. The second one, having a full timeslice, might finish
the operation. When control returns to the first process, confusion
will reign.
The trick is to do something atomic, so that this won't
happen. There are a couple ways to do this. One is to create a
directory instead of a file, the other is to create a symbolic
link. Both operations are defined to be atomic by POSIX/SUS, by
virtue of the fact that they both require invocation of the
corresponding system calls, which are atomic.
Beware of trying to create ANY kind of lock file on an NFS
partition. NFS pretty much eliminates anything like atomicity. If
you're going to create a lock file, make sure you're doing it on a
local partition, such as /tmp.
Netscape/Mozilla uses the symbolic link method for its lockfile (in
spite of the fact that it creates it in the user's home directory,
which may be NFS mounted). When it starts up it creates a file
named for the IP address of the machine it's running on, and the
pid of the creating process. Then it tries to create a symbolic
link named "lock", which points to that file. If this symlink
already exists, link(2) will return an error. In a script this
would work something like
touch /tmp/xxx
ln -s /tmp/xxx /tmp/lockfile 2>/dev/null
ret=$?
rm /tmp/xxx
if [ $ret -ne 0 ];then
echo lockfile already exists
exit 1
else
echo success
fi
If you have procmail installed, another possibility is the
lockfile(1) command that comes with it.
======================================================================
9. How can I convert DOS text files to unix, and vice versa?
Unix text files consist of lines delimited by an LF ("line-feed")
character (ASCII 10). DOS uses the two characters CR LF ("carriage
return", "line feed"; ASCII 13, 10) for the same purpose.
To convert a DOS text into unix text format, the CR characters
(control-M) at the end of a line have to be removed. To create a
DOS text file, the CR character should be added.
A couple ways to remove CR characters:
sed 's/^M$//' dos.txt > unix.txt
tr -d '\r' < dosfile > unixfile
To add them:
sed 's/$/^M/' unix.txt > dos.txt
Note that "^M" in this case is an embedded control character, (CR,
ASCII 13). Many shells allow embedding control characters by
entering ^V first (control-V), resulting in the sequence
^V^M
for entering "^M".
However, zsh, bash or ksh93 allow for:
sed $'s/$/\r/'
There is one special case to be considered: DOS text files
sometimes contain an explicit end-of-file character ^Z (ASCII 26,
or octal 32), which has no correspondent character for unix text
files, where the end-of-file condition is determined
implicitly. To remove that as well as the CR characters:
tr -d '\r\032' < dosfile > unixfile
Note that sed does not understand that notation, but awk does, and
one simple way to do the opposite conversion is
$ awk '{printf "%s\r\n" $0}END{printf "%c", 26}' unixfile > dosfile
This assume a not-quite-ancient awk, in practice anything
but Solaris /bin/awk (use nawk or /usr/xpg4/bin/awk in Solaris).
Finally, your system may come with utilities named something like
dos2unix and unix2dos, or d2u dos2unix fromdos non-standard
utilities, and GNU recode: recode /CRLF
======================================================================
10. How can a shell prompt be set up to change the title of xterm?
Gives escape sequences for xterm. For example, to change the name
of the current window to "XXX" (in bash), do
$ echo -en "\033]2;XXX\007"
or, more portably:
$ printf '%b' '\e]2;XXX\a'
See also "Why doesn't echo do what I want?"
[ 本帖最后由 r2007 于 2005-12-3 16:59 编辑 ]
r2007 回复于：2005-05-25 21:44:42
11. How do I get the exit code of cmd1 in cmd1|cmd2
First, note that cmd1 exit code could be non-zero and still don't
mean an error. This happens for instance in
cmd | head -1
you might observe a 141 (or 269 with ksh93) exit status of cmd1,
but it's because cmd was interrupted by a SIGPIPE signal when
"head -1" terminated after having read one line.
To know the exit status of the elements of a pipeline
cmd1 | cmd2 | cmd3
a. with zsh:
The exit codes are provided in the pipestatus special array.
cmd1 exit code is in $pipestatus[1], cmd3 exit code in
$pipestatus[3], so that $? is always the same as
$pipestatus[-1].
b. with bash:
The exit codes are provided in the PIPESTATUS special array.
cmd1 exit code is in ${PIPESTATUS[0]}, cmd3 exit code in
${PIPESTATUS[2]}, so that $? is always the same as
${PIPESTATUS: -1}.
c. with any other Bourne like shells
You need to use a trick to pass the exit codes to the main
shell. You can do it using a pipe(2). Instead of running
"cmd1", you run "cmd1; echo $?" and make sure $? makes it way
to the shell.
exec 3>&1
eval `
# now, inside the `...`, fd4 goes to the pipe
# whose other end is read and passed to eval;
# fd1 is the normal standard output preserved
# the line before with exec 3>&1
exec 4>&1 >&3 3>&-
{
cmd1 4>&-; echo "ec1=$?;" >&4
} | {
cmd2 4>&-; echo "ec2=$?;" >&4
} | cmd3
echo "ec3=$?;" >&4
`
d. with a POSIX shell
You can use this function to make it easier:
run() {
j=1
while eval "\${pipestatus_$j+:} false"; do
unset pipestatus_$j
j=$(($j+1))
done
j=1 com= k=1 l=
for a; do
if [ "x$a" = 'x|' ]; then
com="$com { $l "'3>&-
echo "pipestatus_'$j'=$?" >&3
} 4>&- |'
j=$(($j+1)) l=
else
l="$l \"\$$k\""
fi
k=$(($k+1))
done
com="$com $l"' 3>&- >&4 4>&-
echo "pipestatus_'$j'=$?"'
exec 4>&1
eval "$(exec 3>&1; eval "$com")"
exec 4>&-
j=1
while eval "\${pipestatus_$j+:} false"; do
eval "[ \$pipestatus_$j -eq 0 ]" || return 1
j=$(($j+1))
done
return 0
}
use it as:
run cmd1 \| cmd2 \| cmd3
exit codes are in $pipestatus_1, $pipestatus_2, $pipestatus_3
======================================================================
12. Why do I get "script.sh: not found"
a. While script starts with "#!/bin/sh" (^M issue)
That's the kind of error that occurs when you transfer a file
by FTP from a MS Windows machine. On those systems, the line
separator is the CRLF sequence, while on unix the line
separator is LF alone, CR being just another ordinary character
(the problem is that it is an invisible one on your terminal
(where it actually moves the cursor to the beginning of the
line) or in most text editors or pagers).
So, if a MSDOS line is "#!/bin/sh", when on a Unix system, it
becomes "#!/bin/sh<CR>" (other names for <CR> are \r, \015, ^M,
<Ctrl-M>).
So, if you run the file as a script, the system will look in
/bin for an interpreter named "sh<CR>", and report it doesn't
exist.
$ sed 'l;d;q' < script.sh
#!/bin/sh\r$
shows you the problem ($ marks the end of line, \r is the CR
character).
b. PATH issue
Sometimes a shell is installed someplace other than /bin or
/usr/bin. For example, a shell which was not part of the OS
installation might be installed into /usr/local/bin. If the
script was written on a machine which had ksh located in
/usr/bin, but was run on a machine where ksh was located in
/usr/local/bin, the shebang line would not resolve correctly.
This is unlikely to occur when using sh. However, if the shell
is bash, zsh, et al, it might be installed in different places
on different machines.
One way around this is to use the env command in the shebang
line. So instead of
#!/bin/sh
use
#!/usr/bin/env sh
Of course, env might itself live in some other directory than
/usr/bin, but it's not likely.
======================================================================
13. Why doesn't echo do what I want?
See also section 0a "Notes about using echo"
The echo command is not consistent from shell to shell. For
example, some shells (bash, pdksh [,?]) use the following
arguments
-n suppress newline at the end of argument list
-e interpret backslash-escaped characters
-E disable interpretation of backslash-escaped characters, even
on systems where interpretation is the default.
However, pdksh also allows using \c to disable a newline at the
end of the argument list.
POSIX only allows \c to be used to suppress newlines, and doesn't
accept any of the above arguments.
ksh88 and ksh93 leave the interpretation of backslash-escaped
characters up to the implementation.
[descriptions of behavior of other shells welcome]
In short, you have to know how echo works in any environment you
choose to use it in, and its use can therefore be problemmatic. If
available, print(1) or printf(1) would be better.
======================================================================
[ 本帖最后由 r2007 于 2005-12-3 17:01 编辑 ]
r2007 回复于：2005-05-25 21:45:18
14. How do I loop through files with spaces in their name?
So, you're going to loop through a list of files? How is this list
stored? If it's stored as text, there probably was already an
assumption about the characters allowed in a filename. Every
character except '\0' (NUL) is allowed in a file path on Unix. So
the only way to store a list of file names in a file is to
separate them by a '\0' character (if you don't use a quoting
mechanism as for xargs input).
Unfortunately most shells (except zsh) and most standard unix text
utilities (except GNU ones) can't cope with "\0"
characters. Moreover, many tools, like "ls", "find", "grep -l"
output a \n separated list of files. So, if you want to
postprocess this output, the simpler is to assume that the
filenames don't contain newline characters (but beware that once
you make that assumption, you can't pretend anymore your code is
reliable (and thus can't be exploited)).
So, if you've got a newline separated list of files in a
list.txt file, Here are two ways to process it:
1-
while IFS= read -r file <&3; do
something with "$file" # be sure to quote "$file"
done 3< list.txt
(if your read doesn't have the "-r" option, either make another
assumption that filenames don't contain backslashes, or use:
exec 3<&0
sed 's/\\/&&/g' < list.txt |
while IFS= read file; do
something with "$file" <&3 3<&-
done
)
2-
IFS="
" # set the internal field separator to the newline character
# instead of the default "<space><tab><NL>".
set -f # disable filename generation (or make the assumption that
# filenames don't contain *, [ or ? characters (maybe more
# depending on your shell)).
for file in $(cat < list.txt); do
something with "$file" # it's less a problem if you forget to
# quote $file here.
done
Now, beware that there are things you can do before building
this list.txt. There are other ways to store filenames. For
instance, you have the positional parameters.
with:
set -- ./*.txt
you have the list of txt files in the current directory, and no
problem with weird characters. Looping through them is just a
matter of:
for file
do something with "$file"
done
You can also escape the separator. For instance, with
find . -exec sh -c 'printf %s\\n "$1" | sed -n '"':1
\$!{N;b1
}
s/|/|p/g;s/\n/|n/g;p'" '{}' '{}' \;
instead of
find . -print
you have the same list of files except that the \n in filenames
are changed to "|n" and the "|" to "|p". So that you're sure
there's one filename per line and you have to convert back "|n"
to "\n" and "|p" to "|" before referring to the file.
======================================================================
15. how do I change my login shell?
See http://www.faqs.org/faqs/unix-faq/shell/shell-differences
Unless you have a very good reason to do so, do not change root's
default login shell. By "default login shell" is meant the shell
recorded in /etc/passwd. Note that "I login as root but don't like
the default shell" isn't a good reason.
The default shell for root is one which will work in single user
mode, when only the root partition is mounted. This is one of the
contexts root works in, and the default shell must accommodate
this. So if you change it to a dynamically linked shell which
depends on libraries that are not in the root partition, you're
asking for trouble.
The safest way of changing root's shell is to login as root and
then
# SHELL=/preferred/shell; export SHELL
# exec <your preferred shell with login flag>
e.g.
# SHELL=/usr/bin/ksh; export SHELL
# exec $SHELL -l
Another possibility is to add something to root's .profile or
.login which checks to see if the preferred shell is runnable, and
then execs it. This is more complicated and has more pitfalls than
simply typing "exec <shell>" when you login though. For example,
one of the libraries that the desired shell relies on might have
been mangled, etc. One suggestion that has been made is
if [ -x /usr/bin/ksh ]; then
SHELL=/usr/bin/ksh; export SHELL
ENV=/root/.kshrc; export ENV
/usr/bin/ksh -l && exit
fi
A safer way is to try to run a command with the preferred shell
before you try to exec it. This will lessen the possibility that
the shell or one of the libraries it depends on has been
corrupted, or that one of the libraries it depends on is not in
the available mounted partitions.
if [ -x /usr/bin/ksh ]; then
/usr/bin/ksh -c echo >/dev/null 2>&1
if [ $? -eq 0 ];then
SHELL=/usr/bin/ksh; export SHELL
ENV=/root/.kshrc; export ENV
/usr/bin/ksh -l && exit
fi
fi
Another common approach is to create another user with UID 0. For
example, FreeBSD systems commonly create an account named toor,
which can be setup however you like. This bypasses the
controversy.
======================================================================
16. When should I use a shell instead of perl/python/ruby/tcl...
a. Portability
In many cases it can't be assumed that perl/python/etc are
installed on the target machine. Many customer sites do not
allow installation of such things. In cases like this, writing
a shell script is more likely to be successful. In the extreme,
writing a pure Bourne shell script is most likely to succeed.
b. Maintainability
If the script is one which serves some important purpose, and
will need to be maintained after you get promoted, it's more
likely that a maintainer can be found for a shell script than
for other scripting languages (especially less used ones such
as ruby, rexx, etc).
c. Policy
Sometimes you're just told what to use :-)
======================================================================
17. Why shouldn't I use csh?
http://www.grymoire.com/Unix/CshTop10.txt
http://www.grymoire.com/Unix/Csh.html#uh-0
http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/
======================================================================
[ 本帖最后由 r2007 于 2005-12-3 17:02 编辑 ]
r2007 回复于：2005-05-25 21:45:48
18. How do I reverse a file?
Non-standard commands to do so are GNU tac and "tail -r". sed
'1!G;h;$!d' is subject to sed limitation on the size of its hold
space and is generally slow.
The awk equivalent would be:
awk '{l[n++]=$0}END{while(n--)print l[n]}'
It stores the whole file in memory.
The best approach in terms of efficiency portability and resource
cosumption seems to be:
cat -n | sort -rn | cut -f2-
"cat -n" is not POSIX but appears to be fairly
portable. Alternatives are "grep -n '^'", "awk '{print NR,$0}'".
Also, nl can be used as
nl -ba -d'
'
i.e. NL as the delimiter.
======================================================================
19. how do I remove the last n lines?
First we need to tell the code how many lines we want to cut
from the bottom of a file.
X=10
Then We can do this:
head -n $(( $(wc -l < file ) - $X )) file >$$ \
&& cat $$ >file && rm $$
The break down:
1) $(wc -l < file)
Find out how many lines are in the file. Need to use
redirection so wc won't print the file name.
2) $(( $lines_in_file - $X ))
Take the output from step one and do some math to find out
how many lines we want to have when all is said and done.
3) head -$lines_when_said_and_done file
extracts all but the unwanted lines from the file,
and >$$ puts those lines into a temp file that has
the name of the pid of the current shell.
4) && cat $$ > file
if everything has worked so far then cat the temp file into
the original file. This is better than mv or cp because it
insures that the permissions of the temp file do not
override with the perms of the original file.
5) && rm $$
Remove the temp file.
AWK solutions:
awk 'NR<=(count-12)' count="`awk 'END{print NR}' file`" file
awk 'NR>n{print a[NR%n]} {a[NR%n]=$0}' n=12 file
awk 'BEGIN{n=12} NR>n{print a[NR%n]} {a[NR%n]=$0}' file
Whenever a line is read, the line that came 12 lines ago is
printed, and then overwritten with the newly read line, using an
rolling array indexed 0..11.
See also question 26. for information about setting awk
variables on the command line.
$SHELL/sed/mv solutions:
L=`wc -l <file`
DL=`expr $L - 11`
sed "$DL,\$d" file
L=`wc -l <file`
DL=`expr $L - 12`
sed "${DL}q" file
sed "`expr \`wc -l <file\` - 12`q" file
sed -n -e :a -e '1,12{N;ba' -e '}' -e 'P;N;D' file
The last solution is basically same algorithm as the rolling
array awk solutions, and shares with them the advantage that
the file is only read once - they will even work in a
pipe. There may be limitations in sed's pattern space which
would make this unusable however.
PERL solution:
perl -ne' print shift @x if @x == 12; push @x, $_ ' file
Using GNU dd:
ls -l file.txt | {
IFS=" "
read z z z z sz z
last=`tail -10 file.txt | wc -c`
dd bs=1 seek=`expr $sz - $last` if=/dev/null of=file.txt
}
======================================================================
20. how do I get file size, or file modification time?
If your system has stat(1), use it.&nbs |
|
