1 LINUX SYSTEMS AT NTV

The computer systems are given in the table below.
to login to Pupa yu need to be in the MPL network (you can use VPN). You can then use our script ztop to see load on each system).
Pupa is license server for intel fortran compiler.
Cameo does admin stuff and mounts all our Jetstor systems (users dont have access to cameo). Jetstor one ( /net/cameo/js) has 8TB of storage and Jetstor two and three ( /net/cameo/js2) has 12 TB of storage.
Both Jetstor systems and /mpl are using raid.

Computer CPU RAM/SWAP Location
Pupa
Xeon X5570, 2 Quad w 2threads/core (2x4x2)= 16 treads
72GB/ 32 GB
Campus
Cameo
Data and disk admin plus web/ftp (Sun)



2 GUIDE LINES FOR CPU USAGE

The optimal load on Pupa it is about 8 one-cpu jobs (one for each core), as more load tends to just become more swapping.
We have no guidelines! Be reasonable!
Some advice:
a) The load on both systems can be monitored by using ztop or the simpler nodeload.
b) If you have many jobs to run, put them into a file and executed sequentially.
c) A job should not take more than a day to run. Or it should be split into sequences that can be tested. That way results can be tested before continuing.
d) For long jobs write the results out with regular intervals (e.g. once an hour).

Cameo does admin stuff and mounts all our Jetstor systems. Jetstor one (/net/cameo/js) has 8TB of storage and Jetstor two and three (/net/cameo/js2 /net/cameo/js3) have 12 TB of storage.

All jobs are niced max after running 30 min CPU.
Computer
max Ram
Max Matlab Thread
Max CPU time
Pupa
30 GB
16
Infinite

3 DISK SPACE AND DATA DIRECTORIES

Outdated description
Disk Purpose quota size Backup
/net/cameo/js
Main user storrage
0.7 TB/ 1TB hard
7TB
raid 6
/net/cameo/js2 Data (SW06)

12 TB
raid 6
/net/cameo/js3 data ( ) + Dave's work dir

12 TB
raid 6
/pupa1


4TB
raid 1
/pupa0
system disk, sys admin, Gerstoft
4 TB
mirrored
You can see how much disk space is used by the command
df -h

You can see how much disk space you use in a directory and subdirectories
du -sh *

Home directories

: This is controlled by setting up quotas for each user. You can see your disk quota. Use
quota -sQ
To see your limits.

Jetstor:
Our three Jetstor systems contains most of our data storage.
Experimental data are stored in /net/cameo/js2/data and /net/cameo/js3/data each containing 12 TB of data

Users directories are on /net/cameo/js/data and contain 8 TB of data

A subdir /net/cameo/js/data/$USER will be created if needed with an appropriate quota (ask mpladmin).
It is usually convenient to make a softlink to that directory in your home dir:
ln -s /net/cameo/js/data/gerstoft js

Quotes are installed on /js with 700GB soft limit and 1TB hard limit. If you have large data requirements these limits can be changed. The hard limit defines the absolute maximum disk space available to a user. The soft limit is more lenient, in that the user is only warned when he exceeds the limit. He can still create new files and use more space for a grace period of 7 days.

4 GRAPHICS

The graphics systems are not well developed under linux. The easiest is to copy the files to your mac or using Expanddrive (see 17)

While most plots is produced in matlab, users can display plots from a remote system using:

eog works with {ani,bmp,gif,ico,jpeg,pcx,png,pnm,ras,svg,tga,tiff,wbmp,xpm}
evince {ps,PS,eps}
xpdf {pdf}
acroread {pdf}

For generating graphics it is usually easier to do it on your laptop. We have installed xfig and imageshark. Imageshark is useful for making movies, see
http://www.imagemagick.org/

5 LONG MATLAB JOBS

Before you start a long job remember to make many small tests--- It is a lot easier to find bugs in small runs!

For a long job you can save the output to mat file and then later read that into matlab for post-processing (recommended). Alternatively you you can print your figures to a file.

MATLAB long jobs: setting an alias is more convenient than remembering the command. Enter the following line in ~/.aliases

alias matlongjob 'nohup nice +19 matlab -nodisplay < \!^ >& \!^.out & echo PID: $! >& \!^.pid'
This puts the pid of the process in filename.m.pid. This is helpful to keep track of jobs when running multiple jobs and you want to kill one of them.

usage: matlongjob filename.m
the command line output is in filename.m.out

6 USER ACCOUNT SETUP

User environment is set up by sourcing system-wide /usr/local/etc/cshrc and /usr/local/etc/login by $USER/.cshrc and $USER/.login.

Users can overwrite system-wide settings or add their own in
~/.cshrc.custom.linux
~/.login.cstom.linux
~/.aliases
~/.aliases.linux

Users are supposed to put their own binaries in subdir 'bin' in their home dir, this dir precedes system dirs in case of name conflict.

By default user's $path is set up to:
/usr/local/intel/fce/10.1.015/bin:.:/mpl/$USER/bin:/mpl/$USER/x86_64/bin:/usr/local/bin:/usr/X11/bin:/usr/bin:/b
in:/usr/local/matlab/bin:/usr/local/gmt/bin

7 Username and password

UCSD Active Directory use your initial+lastname (e.g pgerstoft). UCSD Business Systems (or single sign on) use your UCSD email (e.g gerstoft@ucsd.edu) http://blink.ucsd.edu/technology/network/access/ *************

8 PRINTERS

Spiess Hall: HP Color LaserJet CM3530 MFP mpl-lwwaklabd.ucsd.edu, IP Address:172.16.128.18
printer name on linux systems: lwwaklabd
Spiess Hall: B/W HP LaserJet 4250 mpl-lwwaklab.ucsd.edu, IP Address:172.16.128.16
OAR: oar2.print.ucsd.edu 172.16.127.22 (2nd floor OAR, color)
mpl-lwntv 172.16.128.4 (black & white)

9 Internet

ssh
To avoid being disconnected due to timeout, enter the following lines in ~/.ssh/config on your local computer (mac)

ServerAliveInterval 60
ServerAliveCountMax 5

netbrowser

: Firefox

ftp client is also available. If transferring binaryfiles you have to set the bin flag. Ulike on a Mac. curl is a command line tool for transferring files with URL syntax, supporting FTP, FTPS, HTTP, HTTPS, SCP etc. It works better than classical ftp. I use this to download data:
curl -o X4.2007.08.1-10.19734.openssl ftp://ftp.iris.washington.edu/pub/userdata/Gerstoft/X4.2007.08.1-10.19734.openssl

10 Screen

By using ' screen' you can have a termial running even after you have disconnected. When you login again you can reattach the screen. See e.q
http://www.rackaid.com/resources/linux-screen-tutorial-and-how-to/
To start a new screen, simply type 'screen' at the prompt. You can start a new process in here and detach the screen by pressing Ctrl-a-d. You can logout of the main shell and screen will still be open. To see all open screens, type 'screen -ls'. If you're attached to the screen, it displayes [Attached], else it displays [Detached]. To reattach a detached screen, type ' screen -r <screen name>' You can complete screen name by tab. To exit a screen, just type 'exit' inside the screen.

11 SAGA benchmark:

cd saga/examples/ramgeo/vertical/
saga tc1v1lay ramgeo

saga/ramgeo running tc1v1lay from the inversion workshop
(10000 forward models) on one CPU.
CPU Times:

astartes 9603.1 25 jan 01
turrid 23821.9
heart 18724 s
heart (f77 -fast ) 14812 s
heart (f77 -fast Solaris 2.8) 5297.4 (27 Dec 02)
ark (David B) f77-fast 26094 s
sundial 13046.9 25 jan
baby 2368.8 s (g77)
baby 1771 s ifc (22 Jan 03)
baby 1434.4 (ifc -O -axW -zero -w, 2 Oct 03)
baby 1475 (ifort -O -axW -u -w -posixlib -Vaxlib -CB, Feb 06)
pupa 1290.6 (ifc -O -axW -zero -w, 2 Oct 03)
occam (Mac G4 1.5GHz laptop) 4159 s (xlf -O )
G5 2 GHz Mac 1700 (xlf -O)
chenfen laptop(Mac intel ) 1237 (ifort aug 06)
Peter's laptop (Mac pro book 2.33 Ghz Core 2 duo ) 675 (ifort 9)
Peter's laptop (Mac pro book 2.33 Ghz Core 2 duo ) 637 (ifort 10.1) 27Mar 07
Mac workstaion 8GB, 3GHz 2xdual Xeon 531
Nerite 2Xquad 568 (ifort v10) 2008
Nerite 2Xquad 575 (ifort v11) June2009
Caglars laptop (Mac pro book 2.5 Ghz Core 2 duo , ifort 11) 521 June09
Peter laptop (Mac pro book 3.06 Ghz Core 2 duo , ifort 10) 434 June09
Peter laptop (Mac pro book 3.06 Ghz Core 2 duo , ifort 11) 425 July09
Peter laptop (Mac pro book 3.06 Ghz Core 2 duo , ifort 11.1 Mac10.6) 418 Oct09/ Nov10
Peter laptop (Mac pro book 3.06 Ghz Core 2 duo , ifort 12 Mac10.6) 415 Nov10
Pupa (2Xquad) ifort v11 478 Feb10
Pupa (2Xquad) ifort v12 450 Nov10
Peter Laptop (Mac book retina 2.6 Ghz Intel Cre i7; ifort 12 Mac 10.8), 303 s Aug 2012
Peter Laptop (Mac book retina 2.6 Ghz Intel Core i7; ifort 12 Mac 10.9), 271 s Nov 2013
Peter Laptop (Mac book retina 2.6 Ghz Intel Core i7; gfortran 5.2 Mac 10.11), 441 s Nov 2015

12 Matlab bencmark

mean(bench(10))
LU FFT ODE Sparse 2D 3D
Nerite 0.1350 0.1834 0.1486 0.3621 0.3501 1.0483
Pupa 0.0339 0.0748 0.1276 0.1713 0.3933 1.4689
Peter Laptop 0.0826 0.2095 0.1642 0.2949 0.6235 0.8768
Peter laptop(macbook retina, 2.6GHz i7, 16 GB 2012) 0.0859 0.1107 0.1332 0.1338 0.5291 0.6520 (Aug 2012)
Wenyuan laptop (core 2 duo 2.5GHz 2008) .22 .28.23 .29 .8 .9
Olivier laptop (2010 core2 duo 3) .06 .11.16 .28 .70 .81
Zhao laptop (2012 13inch i5 2.5GHz) .18 .19 .16 .18 .82 .80
Peter laptop(macbook retina, 2.6GHz Intel Core i7, 16 GB 2013) 0.036 0.073 0.072 0.09 0.29 0.47 (Nov 2013)
Peter laptop(macbook retina, 2.6GHz Intel Core i7, 16 GB 2013)0.087 0.057 0.049 0.10 0.37 0.79
Peter Imac (3.4GHz Intel Core I5) 0.039 0.077 0.069 0.097 0.30 0.36 (Jan 2014)

13 WWW upload

The MPL webpages resides on mplwd.ucsd.edu
I develop my webpages locally and then upload it all, eg:
scp -r *html pgerstoft@mplwd.ucsd.edu:WWW

where the WWW is a softlink. Created by
ln -s
The username and password on mplwd is the "AD" (email) username and passwd.

14 FTP

The ftp server is ftp.mpl.ucsd.edu
I have a softlink
ln -s /var/spool/ftp/pub/pgerstoft ftp

14 Intel Fortran Compiler

On the unix systems the Intel fortran compiler is invoked by ifort
We have a fortran group license. It resides on pupa. You can run ifort on your local computer povided the compiler is installed. All IPs in mpl vpn pool have access to the license, so start vpn first. The license is from 2010 and has not been renewed, so it does not work on Mac OSX 10.8 and higher.

15 Matlab laptop installation

For matlab on your laptop you can either have a standalone license or a campus netweork site license. The latter is cheaper but requires that you are on the internet. Since I am always online, I just use that solution.

For the campus-site-license installation, Mplhelp can provide you the File Installation Key (FIK) and network.lic file. I put the network.lic in ~/Documents/MATLAB/
The license file resides here: /Applications/MATLAB_R2010b.app/licenses
I currently switch between my standalone license (with just SP toolbox) and my UCSD internet license (all tooboxes) by cp'ing the relevant file to license.lic
Download the matlab from matlab.com under account services when you log-in.
Then follow these steps:
http://www.mathworks.com/support/solutions/en/data/1-3UVPJX/

16 VPN

Get vpn ( see blink) and ask mplhelp to be added to mpl vpn pool.

17 Expandrive


It might be useful to have expandrive or macfusion on you Mac that will enable you to mount linux disk on your mac and access the files as on a mac. Note, that this should not be used alone, but also with xterm.

18 some mac notes

The reason for the slow wake up is because of the default hibernate mode on macs. By default, it goes into "deep sleep" after 1 hour of sleep, meaning it writes the RAM contents to disk and shuts off the RAM. So even if you use only 2GB of the total 16GB, it creates a 16GB file on disk. Look at this, for instance:

ls -lh /var/vm/sleepimage
So when you open the lid after a while, it has to read 16GB and restore the contents back to RAM. In our older laptops, we didn't notice this because it was only 4GB of memory that had to be read and we also expected it to be slow.

To fix this, make the following changes:
sudo pmset -a hibernatemode 0
sudo rm /var/vm/sleepimage

and this should fix the problem. With this change, the RAM is always on, even when in sleep mode. The only "drawback" of this is that you can't use standby mode (i.e., keep it unused on battery for several weeks) and if you allow the battery to drain fully, then it won't restore the previous state. It'll do a full restart. This should not be a concern, because we rarely use it away from a power source and such a situation is very rare.

To go back to the original mode, just do:
sudo pmset -a hibernatemode 3

The sleepimage file will be automatically written again. I've tried it for a few days and had no problems so far.

screen sharing on macs

it is located in /System/Library/CoreServices/Screen Sharing.app or you can get to it via finder - go - connect to server:
vnc://hostnanme <
to allow incoming screen sharing connections go to:
system preferences - sharing - remote management - allow access for
add user account to a list here and click options button to set control option.

Adobe

To download Acrobat, login with your UCSD Active Directory (AD) username and password at: https://software-web.ucsd.edu:10081 Mplhelp might need to add you to the user list, and will provide a License file.

Matlab

You can download MATLAB iso file from ACMS Software Distribution Matlab download site:
https://matlab.ucsd.edu