Sunday, February 25, 2007

Build up home data center and backup policy

Last a few days I re-organized/cleanup my large hard disks to build up a center repository. All legacy data, reference information and current data are stored on the server which is the only data storage. The advantage is that there is no need to synchronize data and it simplify backup. And all data can be accessed anywhere as well.

A new backup policy is defined. It's in fact as important as the hard disk. :) Basically I want
1. robust data management
2. easy and flexible to use, accessible anywhere
3. easy to automate backup
4. searchable by desktop search engine

The general backup rule is:
1. all fixed data are stored on the external hard disk with read only permission.
2. all dynamic data are stored on another partition on external hard disk.
3. fixed data are backup to DVD with snapshots, dynamic data are weekly (monthly for media) archived and ftp to another computer.

Backup of dynamic data can be automated. I am going to write a script to do that automatically. All data is robust because they have at least one extra copy elsewhere.

All data on server are mapped to windows network drive. It's easy to use and new google desktop can index it! And it is really indexing now! :)

Resource now I have
/dev/hda1 system disk 10G
/dev/hdc5 temporary data disk 10G, used for Bittorrent as well
/dev/sda1 user data 15G
/dev/sda2 repository for fixed data and media 205G NTFS
/dev/sda3 shared data 15G

I bought an external Philips DVD writer last week. At first glance it looks not so good but now I like it very much. :) But I didn't try to burn DVD on Linux yet.

By the way, some often used commands for CVS under linux. Tortoise doesn't work with network drive.

CVSROOT=:ssh:sunwei@192.168.11.99:/home/cvs
(if on the same computer, simply set CVSROOT=/home/cvs)
cvs -q checkout [-r tag] -P ejpgl
cvs -q import ejpgl tcvs-vendor tcvs-release
cvs -q commit *.c
cvs -q tag -c TAGNAME

The last thing is that RealView can't open multiple MP3 files. I installed VideoLAN instead following instructions on VideoLAN site in yum approach. But in 'yum update' it's always error that it can't access some server. I use 'yum clean‘ and 'yum install video-lan' without update and it works.

Thursday, February 8, 2007

Upgrade to Fedora 4

Although RH7.2 works well, it doesn't support some new applications like VLC. I upgrade it into Fedora 4 in last a few days. Another reason I choose Fedora is that it suggests Pentium 500 and 256MB memory. That fits my DELL server well which is PIII 800 with 256MB RDRAM.

There is a great guide about how to set up Fedora 4, step by step, very detailed, http://stanton-finley.net/fedora_core_4_installation_notes.html. I downloaded the image from a mirror site http://ftp.uni-koeln.de/mirrors/fedora/linux/core/

The configuration is almost the same to RH7.2 while I use the whole disk for Fedora. It's because that I noticed some discussion about the problems to have Linux and Windows on the same disk. I also choose all graphical administration tools. After it starts, I disabled unused daemons according to Stanton.

CVS:
It's almost the same to that on RH7.2 while now the cvs is already exists in /etc/xinetd.d. You only need to enable it.

Samba:
You don't need to edit /etc/samba/smb.conf. Now it can be done via graphical tool. The other difference is that user access is not limited to his home directory. You can add any share directory and decide who can access it. After saving, restart smb by 'service smb restart'.

SSH and X-windows:
SSH is enabled by default. To set up remote X-windows, it's the same to that in RH7.2

FTP and web server:
Just enable vsftpd and httpd.

I also connect an external hard disk with NTFS to Fedora. It works well! To do that, we need to download the NTFS package from http://www.linux-ntfs.org/. Note we should download the package for Fedora not old Linux. Then rpm -ivh kernel-module-ntfs-2.6.11-1.1369_FC4-2.1.22-0.rr.6.0.i686.rpm. To be able to be accessed by users other than root, use 'mount -t ntfs -r -o umask=0222 /dev/sda2 /home/repository/'. Done!

Before I found the umask trick, all files are only accessible from root. I made a mistake to use 'chmod -R o+x o+r *' to add permission. I don't know if it would impact the NTFS system. I need to run filesystem check later.

I installed Xilinx EDK8.1 on it. The strange thing is that ISE8.1 always doesn't accept the registration ID which is correct. I try ISE webpack 8.1 later.

Anyway, my computing center is upgraded and now very convenient to use. :P

Monday, February 5, 2007

CVS Server

When I started to inspect the code, I realized that I need a CVS server as well. First check if there is any free CVS hosting site. Well, there are some, but $30 per month!

It's not difficult to setup a CVS server. I found a good paper at http://personal.vsnl.com/sureshms/linuxindex.html.

1. create cvs user and group
2. CVSROOT=/home/cvs; export CVSROOT add it into profile
3. cvs -d /home/cvs init
4. add /etc/xinetd.d/cvs
5. service xinetd restart

As it uses sshd, it's not necessary to start telnetd.

Done!

On client side, I use tortoiseCVS. Set home :ssh:sunwei@192.168.11.99:/home/cvs. First create a new directory. Use CVS->Make New Module, repository folder->/home/cvs, modulename->directory name. Afterwards you can add files into it.

Everything is now saved at /home/cvs. To backup I think just simply tar /home/cvs. It's easy. :)

Another problem of the Linux server is that I need to set DNS server address for it. It's /etc/resolv.conf. add 'nameserver 192.168.11.1' then it's fine.

service cvspserver
{
disable = no
socket_type = stream
protocol = tcp
wait = no
user = cvs
group = cvs
log_type = FILE /var/log/xinetdlog
server = /usr/bin/cvs
server_args = -f --allow-root=/home/cvs pserver
log_on_success += USERID DURATION
log_on_failure += HOST USERID
}

Sunday, February 4, 2007

Start to build my mini computing center

I experienced top-level IT infrastructure during my last internship. It's a kind of combination of Linux server clusters and Windows desktop. I like to build up my mini computing center at home.

Today I installed a Redhat 7.2 on the DELL. It's quite successful. The DELL server is 800Mhz system, though, it runs quite fast with RH7.2. I setup Samba, ftp, sshd, X-server, web services on that.

The installation disk is from the book "Red Hat Linux, the complete reference 2nd" by Richard Petersen. The fist trick is type of installation. I should choose customized otherwise it use the whole disk for Linux. The second is to partition the disk. It requires a separate swap partition which need to be at least the same the memory size. Then it goes. Of course, these used packages need to be installed. Sometimes Samba is called something like windows file sharing.

Samba
Go to /etc/samba. Edit smb.conf. Samba use a password file other than /etc/passwd. You can make it by

cat /etc/passwd | mksmbpasswd.sh > /etc/samba/smbpasswd
chmod 600 /etc/samba/smbpasswd

However, Samba can only get username but not password from it. You need to run smbpasswd username to set password. After that, use command 'service smb start/restart'' to invoke.

To make it start automatically during booting, you need command 'chkconfig --level 35 smb on'. It enables smb during booting. Except for ftp server, sshd and httpd are both enabled by 'çhkconfig --level 35' because they are enabled in rc.d while ftpd is enabled in xintd.d.

There is a problem with cygwin. It can't write to Samba drive. It looks a common problem. Someone write a description but it doesn't work. Search "cygwin samba permission".

Ftp server
Simply use command 'chkconfig wu-ftpd on ; service xinted restart'. Ftp is enabled via xinetd. You can find the configuration file /etc/xinetd.conf/wu-ftpd. What chkconfig do is to remove the last line 'disable=yes'.

Sometimes it's very slow to connect to it from a client. But it's fast afterwards.

sshd & X-server
sshd can be enabled by 'chkconfig --level 35 sshd on'. Use putty to connect to it. To setup X-server, I found a good reference http://www.yolinux.com/TUTORIALS/LinuxTutorialMicrosoftWindowsNetworkIntegration.html. Basically it needs three steps.

1) edit /etc/X11/xdm/Xaccess, change from '#* any host can get a login window' to '* any host can get a login window'
2) /etc/X11/xdm/xdm-config, change from '!DisplayManager.requestPort 0' to 'DisplayManager.requestPort 0'
3) /etc/X11/gdm/gdm.conf, change from '[xdmcp] Enable=false' to '[xdmcp] Enable=true'.

And init 3 ; init 5 to restart X-windows.

Apache
Simply use 'chkconfig --level 35 httpd on'. The home directory is /var/www. There are already some files in. I didn't try Tcl yet.

Now I can use my notebook to control everything on Linux server! Excellent!

The only problem of RH7.2 is that ISE/EDK8.1 requires RH Enterprise Linux 3 WS or 4 WS, which are equivalent to RHL9 or Fedora 3 (from wiki). I have to re-install it when Internet is available.

Enclosing smb.conf
# This is the main Samba configuration file. You should read the
# smb.conf(5) manual page in order to understand the options listed
# here. Samba has a huge number of configurable options (perhaps too
# many!) most of which are not shown in this example
#
# Any line which starts with a ; (semi-colon) or a # (hash)
# is a comment and is ignored. In this example we will use a #
# for commentry and a ; for parts of the config file that you
# may wish to enable
#
# NOTE: Whenever you modify this file you should run the command "testparm"
# to check that you have not made any basic syntactic errors.
#
#======================= Global Settings =====================================
[global]

# workgroup = NT-Domain-Name or Workgroup-Name
workgroup = HOME

# server string is the equivalent of the NT Description field
server string = RHServer

# This option is important for security. It allows you to restrict
# connections to machines which are on your local network. The
# following example restricts access to two C class networks and
# the "loopback" interface. For more examples of the syntax see
# the smb.conf man page
hosts allow = 192.168.11. 127.

# if you want to automatically load your printer list rather
# than setting them up individually then you'll need this
printcap name = /etc/printcap
load printers = yes

# It should not be necessary to spell out the print system type unless
# yours is non-standard. Currently supported print systems include:
# bsd, sysv, plp, lprng, aix, hpux, qnx
printing = lprng

# Uncomment this if you want a guest account, you must add this to /etc/passwd
# otherwise the user "nobody" is used
; guest account = pcguest

# this tells Samba to use a separate log file for each machine
# that connects
log file = /var/log/samba/%m.log

# Put a capping on the size of the log files (in Kb).
max log size = 100

# Security mode. Most people will want user level security. See
# security_level.txt for details.
security = user

# Use password server option only with security = server
# The argument list may include:
# password server = My_PDC_Name [My_BDC_Name] [My_Next_BDC_Name]
# or to auto-locate the domain controller/s
# password server = *
; password server =

# Password Level allows matching of _n_ characters of the password for
# all combinations of upper and lower case.
; password level = 8
; username level = 8

# You may wish to use password encryption. Please read
# ENCRYPTION.txt, Win95.txt and WinNT.txt in the Samba documentation.
# Do not enable this option unless you have read those documents
encrypt passwords = yes
smb passwd file = /etc/samba/smbpasswd

# The following is needed to keep smbclient from spouting spurious errors
# when Samba is built with support for SSL.
; ssl CA certFile = /usr/share/ssl/certs/ca-bundle.crt

# The following are needed to allow password changing from Windows to
# update the Linux sytsem password also.
# NOTE: Use these with 'encrypt passwords' and 'smb passwd file' above.
# NOTE2: You do NOT need these to allow workstations to change only
# the encrypted SMB passwords. They allow the Unix password
# to be kept in sync with the SMB password.
; unix password sync = Yes
; passwd program = /usr/bin/passwd %u
; passwd chat = *New*password* %n\n *Retype*new*password* %n\n *passwd:*all*authentication*tokens*updated*successfully*

# Unix users can map to different SMB User names
; username map = /etc/samba/smbusers

# Using the following line enables you to customise your configuration
# on a per machine basis. The %m gets replaced with the netbios name
# of the machine that is connecting
; include = /etc/samba/smb.conf.%m

# This parameter will control whether or not Samba should obey PAM's
# account and session management directives. The default behavior is
# to use PAM for clear text authentication only and to ignore any
# account or session management. Note that Samba always ignores PAM
# for authentication in the case of encrypt passwords = yes

; obey pam restrictions = yes

# Most people will find that this option gives better performance.
# See speed.txt and the manual pages for details
socket options = TCP_NODELAY SO_RCVBUF=8192 SO_SNDBUF=8192

# Configure Samba to use multiple interfaces
# If you have multiple network interfaces then you must list them
# here. See the man page for details.
; interfaces = 192.168.12.2/24 192.168.13.2/24

# Configure remote browse list synchronisation here
# request announcement to, or browse list sync from:
# a specific host or from / to a whole subnet (see below)
; remote browse sync = 192.168.3.25 192.168.5.255
# Cause this host to announce itself to local subnets here
; remote announce = 192.168.1.255 192.168.2.44

# Browser Control Options:
# set local master to no if you don't want Samba to become a master
# browser on your network. Otherwise the normal election rules apply
; local master = no

# OS Level determines the precedence of this server in master browser
# elections. The default value should be reasonable
; os level = 33

# Domain Master specifies Samba to be the Domain Master Browser. This
# allows Samba to collate browse lists between subnets. Don't use this
# if you already have a Windows NT domain controller doing this job
; domain master = yes

# Preferred Master causes Samba to force a local browser election on startup
# and gives it a slightly higher chance of winning the election
; preferred master = yes

# Enable this if you want Samba to be a domain logon server for
# Windows95 workstations.
; domain logons = yes

# if you enable domain logons then you may want a per-machine or
# per user logon script
# run a specific logon batch file per workstation (machine)
; logon script = %m.bat
# run a specific logon batch file per username
; logon script = %U.bat

# Where to store roving profiles (only for Win95 and WinNT)
# %L substitutes for this servers netbios name, %U is username
# You must uncomment the [Profiles] share below
; logon path = \\%L\Profiles\%U

# Windows Internet Name Serving Support Section:
# WINS Support - Tells the NMBD component of Samba to enable it's WINS Server
; wins support = yes

# WINS Server - Tells the NMBD components of Samba to be a WINS Client
# Note: Samba can be either a WINS Server, or a WINS Client, but NOT both
; wins server = w.x.y.z

# WINS Proxy - Tells Samba to answer name resolution queries on
# behalf of a non WINS capable client, for this to work there must be
# at least one WINS Server on the network. The default is NO.
; wins proxy = yes

# DNS Proxy - tells Samba whether or not to try to resolve NetBIOS names
# via DNS nslookups. The built-in default for versions 1.9.17 is yes,
# this has been changed in version 1.9.18 to no.
dns proxy = no

# Case Preservation can be handy - system default is _no_
# NOTE: These can be set on a per share basis
; preserve case = no
; short preserve case = no
# Default case is normally upper case for all DOS files
; default case = lower
# Be very careful with case sensitivity - it can break things!
; case sensitive = no

#============================ Share Definitions ==============================
[homes]
comment = Home Directories
browseable = yes
writable = yes
valid users = %S
create mode = 0666
directory mode = 0777
# If you want users samba doesn't recognize to be mapped to a guest user
; map to guest = bad user


# Un-comment the following and create the netlogon directory for Domain Logons
; [netlogon]
; comment = Network Logon Service
; path = /usr/local/samba/lib/netlogon
; guest ok = yes
; writable = no
; share modes = no


# Un-comment the following to provide a specific roving profile share
# the default is to use the user's home directory
;[Profiles]
; path = /usr/local/samba/profiles
; browseable = no
; guest ok = yes


# NOTE: If you have a BSD-style print system there is no need to
# specifically define each individual printer
[printers]
comment = All Printers
path = /var/spool/samba
browseable = no
# Set public = yes to allow user 'guest account' to print
guest ok = no
writable = no
printable = yes

# This one is useful for people to share files
;[tmp]
; comment = Temporary file space
; path = /tmp
; read only = no
; public = yes

# A publicly accessible directory, but read only, except for people in
# the "staff" group
;[public]
; comment = Public Stuff
; path = /home/samba
; public = yes
; writable = yes
; printable = no
; write list = @staff

# Other examples.
#
# A private printer, usable only by fred. Spool data will be placed in fred's
# home directory. Note that fred must have write access to the spool directory,
# wherever it is.
;[fredsprn]
; comment = Fred's Printer
; valid users = fred
; path = /home/fred
; printer = freds_printer
; public = no
; writable = no
; printable = yes

# A private directory, usable only by fred. Note that fred requires write
# access to the directory.
;[fredsdir]
; comment = Fred's Service
; path = /usr/somewhere/private
; valid users = fred
; public = no
; writable = yes
; printable = no

# a service which has a different directory for each machine that connects
# this allows you to tailor configurations to incoming machines. You could
# also use the %U option to tailor it by user name.
# The %m gets replaced with the machine name that is connecting.
;[pchome]
; comment = PC Directories
; path = /usr/local/pc/%m
; public = no
; writable = yes

# A publicly accessible directory, read/write to all users. Note that all files
# created in the directory by users will be owned by the default user, so
# any user with access can delete any other user's files. Obviously this
# directory must be writable by the default user. Another user could of course
# be specified, in which case all files would be owned by that user instead.
;[public]
; path = /usr/somewhere/else/public
; public = yes
; only guest = yes
; writable = yes
; printable = no

# The following two entries demonstrate how to share a directory so that two
# users can place files there that will be owned by the specific users. In this
# setup, the directory should be writable by both users and should have the
# sticky bit set on it to prevent abuse. Obviously this could be extended to
# as many users as required.
;[myshare]
; comment = Mary's and Fred's stuff
; path = /usr/somewhere/shared
; valid users = mary fred
; public = no
; writable = yes
; printable = no
; create mask = 0765