IT Support Blog » Linux

Understanding swap files in Linux

aionman — Wed, 29 Jun 2011 05:34:40 +0000

http://distilledb.com/blog/archives/date/2009/02/22/swap-files-in-linux.page

Understanding swap files in Linux

Summary If you run anything close to a modern operating system, you almost certainly interact with a swap file. You might be familiar with the basics of how these work: they allow your OS to prioritize more frequently-used pages in main memory and move the less frequently-used ones to disk. But there’s a lot going on underneath the covers. Here’s a simple guide to swap files in Linux.

To appease some of my hungrier applications and support heftier development efforts, I recently upgraded the memory on my system from 4 GiB to 8 GiB. That gave me occasion to tinker around with the swapping behavior of my system and check things out.

If you run anything close to a modern operating system, you almost certainly interact with a swap file. You might be familiar with the basics of how these work: they allow your OS to prioritize more frequently-used pages in main memory and move the less frequently-used ones to disk. But there’s a lot going on underneath the covers. Here’s a simple guide to the theory and practice of swap files in Linux, and how you can tweak things for your benefit.

An abstract memory model

It’ll be useful to have a mental model of the way memory works in general¹, so we’ll start from the basis of a simple one here.

In general, all computers have access to physical memory, where the actual bits are manipulated and stored for use. Most modern operating systems present physical memory to higher-level applications as an abstraction calledvirtual memory. This allows applications to see memory as if it were a contiguous block, even though the underlying physical memory may theoretically be taken from many arbitrary, heterogeneous sources — multiple memory chips, flash memory, disk drives, and so on.

The memory manager divides virtual memory into chunks of identical size called pages. A page is the smallest amount that the OS will allocate in response to requests from programs. It is also the smallest unit of transfer between main memory and any other location, such as a hard disk. The size of a page is usually fixed by the operating system’s kernel².

Applications see virtual memory as a contiguous resource divided into units called pages. A typical page size for a modern desktop system is about 4 kilobytes.

As pages are allocated to applications, they are assigned pages in the physical memory space through a special mapping called address translation. Applications don’t know where they are in the physical space; they see only the pages they use.

Address translation maps virtual pages onto physical pages. This mapping is transparent to applications.

The memory manager knows how to aggregate different backing stores to provide the abstraction of contiguous virtual memory. By updating the address translation mechanism so that a virtual page always points to the correct physical page, the memory manager may move pages around in physical memory without directly impacting applications.

The backing stores for pages can be quite heterogeneous.

Wide variation in different kinds of physical memory

Not every backing store displays the same storage and speed characteristics. If different types of physical memory display different characteristics, the memory manager can exploit these differences to optimize system performance. It can make intelligent decisions about which pages should go where.

Consider the difference between main memory and a hard drive, for example.

characteristic	storage medium
	on disk		in main memory
	absolute	relative	absolute	relative
access time	5ms	1×	10ns	500,000× faster
peak transfer rate	80 MB/s	1×	8 GB/s	100× faster
storage space	100 GB	25× larger	4 GB	1×
price/GB storage	$0.60/GB	20× cheaper	$12/GB	1×

Some comparisons of a typical hard drive and typical main memory on a consumer-grade desktop.

Because storage on disk-backed file systems does indeed have different characteristics than storage in main memory, there’s significant room to optimize depending on the characteristics of how pages are accessed. As the table shows, hard disks are several orders of magnitude slower and less efficient at retrieving data than main memory. Thus, the memory manager needs to carefully balance the demand for memory against the different kinds of supply.

Flavors of pages

If all pages were of the same kind, deciding which pages should go where would be a simpler decision. For example, one strategy is to make the access time quickest for the pages that are accessed the most frequently. Unfortunately, it’s not just the types of memory that are heterogeneous, but also the contents of the pages that are stored there. On Linux, there are four different kinds of pages.

Kernel pages. Pages holding the program contents of the kernel itself. Unlike the other flavors, these are fixed in memory once the operating system has loaded and are never moved.
Program pages. Pages storing the contents of programs and libraries. These are read-only, so no updates to disk are needed.
File-backed pages. Pages storing the contents of files on disk. If this page has been changed in memory (for example, if it’s a document you’re working on), it will eventually need to be written out to disk to synchronize the changes.
Anonymous pages. Pages not backed by anything on disk. When a program requests memory be allocated to perform computations or record information, the information resides in anonymous pages.

When the in-memory version of a page is the same as the one on disk, we say that the page is clean; the contents are the same. But sometimes the contents of a page have been updated since the last time they were read. When this happens, the page becomes dirty.

A clean page can be repurposed for something else easily; no updates need to be made, and the page can simply be recycled. But a dirty page has to be written back to disk before it can be used again. For file pages, this is an expensive operation, so the kernel tries to avoid the overhead of flushing back to disk when it can.

For anonymous pages, there’s a different problem. Effectively, they’re always dirty: the very act of creating the anonymous page means that there is now data that is in memory which isn’t in disk. If the kernel wants to use anonymous pages for something else, it must first reclaim them. But anonymous pages have no files to back them. How can you flush something back to disk when there’s nowhere to flush it to?

Swap files

The use of swap can resolve many of these issues. Swap is a disk-backed area that’s treated as an extension of main memory. It serves as a holding area for pages that have been evicted by the kernel. Let’s use an illustrative example to show how swap files help make memory work better.

Legend for the next few diagrams. Unused pages have dotted borders; dirty pages have an alert symbol; and anonymous and file pages are colored orange and green, respectively.

When a moderately loaded system gets additional requests for memory, the kernel generally draws from the pool of free pages first to fulfill these requests. If there are few free pages remaining, the kernel tries to flush clean pages to make room for the new requests.

The kernel prefers to go after unused pages first.

If the clean pages also become depleted, the kernel is forced to clean a dirty page and then flush it. This is an expensive operation. For this reason, the kernel tries to maintain at least some clean pages all the time.

When the ratio of anonymous pages to dirty pages is high and the number of clean pages is low, the kernel is running out of memory. Without swap, this situation will require a number of costly disk writes. Consider a request for allocation when a number of dirty pages are already present.

Without swap space, it’s easier for systems to get overloaded.

In the figure above, a request for two anonymous pages has come in. There are no more unused pages, so the kernel must drop one of the existing pages to satisfy the request. The kernel can use one page freely: the single clean file page in slot 6.

But to allocate the second page, the kernel now has to flush one of the dirty file pages (in slots 1, 3, or 4) back to disk to make room. It cannot move the page in slot 5 anywhere, because it is anonymous and has no backing store; there’s nowhere else to put it. Even if this page has not been used in a very long time, it must still occupy space in memory until the process using it has released the page.

When space is tight and there’s no swap the kernel must make room by cleaning dirty pages and freeing them. In this example, the kernel is forced to clean page #1 back to disk to make room for the second allocated page.

Without swap, the kernel gets boxed into this unfortunate corner more easily.

With swap, however, the kernel gets an additional tool to use in its arsenal. Instead of being forced to clean one of the dirty pages, it can instead evict one of the anonymous pages to the swap region.

When swap is available, the kernel doesn’t need to clean dirty pages, and can instead move anonymous pages to swap.

As in the earlier non-swap scenario, the kernel use can use the clean page in slot 6 for the first requested page. It is allocated and the clean page is dropped.

For the second requested page, the kernel must no longer clean a dirty page to make room. Instead, it can simply flush one of the anonymous pages to the swap region. The code required to do this is generally very simple and significantly less complex than cleaning a dirty page, and the kernel prefers swapping to cleaning dirty file-backed pages.

Optimizing your swap settings

Linux provides a number of ways to interact with your swap. Two are detailed here:

Aggressiveness of swapping
Adding and removing additional swap containers

Controlling aggressiveness of swapping

The more aggressively the kernel swaps, the more efficiently existing memory can be put to use. Pages that look like they’re not being used will be swapped out rapidly. If the kernel swaps too often, though, applications that were using those pages will take longer to become responsive again as the kernel swaps their memory back into main memory.

For a desktop user, responsiveness of applications can be important, so an aggressive swap may not be desirable, even if it results in less efficient use of memory. For servers and other non-interactive systems, more aggressive swapping may be appropriate and acceptable.

On Linux, this careful balancing act can be configured to meet your personal preferences. The kernel swaps out pages with a zealousness controlled by a swappiness setting.

Swappiness is an integer that ranges from 0 to 100, and indicates the degree to which the kernel favors swap space over main memory. Higher swappiness means that the kernel will move things to swap more frequently. Lower swappiness means that the kernel tries to avoid using swap. A swappiness of zero causes the kernel to avoid swap for as long as possible.

Ubuntu and several other Linux distributions have a default swappiness of 60. You can check your swap setting by reading a /proc/sys value:

$ cat /proc/sys/vm/swappiness
60

To temporarily modify your swappiness, simply edit this value:

$ sudo sysctl vm.swappiness=40
vm.swappiness = 40

This setting lasts until reboot or you change it again with another sysctl vm.swappiness invocation. To make this setting take effect on every reboot, edit your /etc/sysctl.conf configuration file.

$ gksudo gedit /etc/sysctl.conf

Find the vm.swappiness line; if none exists, add it.

vm.swappiness = 40

Adding swap containers

Modern operating systems generally have either a swap partition or a swap file. In a swap partition, part of the hard drive is sliced off and becomes dedicated to swap. A swap file is just an ordinary file that holds up to its file size in swapped pages.

A swap file is considerably less complicated than a swap partition to establish. There is no speed difference between the two³, so swap files are favorable in this respect. However, if you want to be able to hibernate or suspend your computer, using a swap partition is required in some cases. (These suspend/hibernate managers usually cannot handle writing to an active file system.)

Making a new swap file is a simple process. In this example, we’ll make a 2 GiB swap file and make it available to the system as additional swap space. We’ll use primary.swap as the name of the example swap file, but there is nothing special about the name of the file or its extension. You may use anything you wish.

First, we need to create the swap file itself. We’ll use a stream of zeroes as the input source (if=/dev/zero), and write it out to a file named primary.swap in the /mnt directory (of=/mnt/primary.swap). We will write 2048 (count=2048) blocks each 1 MiB in size (bs=1M). Depending on the speed of your hard disks, this may take a little while.

$ sudo dd if=/dev/zero of=/mnt/primary.swap bs=1M count=2048
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 30.1085 s, 71.3 MB/s

Next, we need to format this file and prepare it for use as a swapping space. The mkswap utility sets up a swap area on a device or file.

$ sudo mkswap /mnt/primary.swap
Setting up swapspace version 1, size = 2097148 KiB
no label, UUID=7be2b3b6-83b0-4afd-8537-197cf12f8c59

After formatting it, the swap can now be added to our system. Use the swapon utility to activate the swap region.

$ sudo swapon /mnt/primary.swap

You can verify that your swap space is now 2 GiB larger.

$ cat /proc/meminfo | grep SwapTotal
SwapTotal:     2097144 kB

Your changes will be lost at reboot, so if you want to make them permanent we’ll need to edit your filesystem table in /etc/fstab.

$ gksudo gedit /etc/fstab

Now add your swap file to the list of filesystems to mount at boot by appending a line to the file.

/mnt/primary.swap  none  swap  sw  0 0

Removing a swap file

Removal works much the same way, but in reverse. If you’ve added your swap to the /etc/fstab list, you need to remove it here first.

To disable your running swaps, run the swapoff utility. You can either specify the swap you’d like to disable, or use the -a parameter.

$ sudo swapoff /mnt/primary.swap

When you disable swap, you force the kernel to clean every page on the swap and/or push it back to main memory. If there is not enough space to squeeze everything in, you may receive out of memory errors from the kernel, so use this judiciously.

Conclusion

Swap files are an essential part of the memory-management modules of operating systems. In Linux, adding and removing swap partitions and files is simple, and you can control how the kernel interacts with swap through configurable parameters. Through the use of these and other techniques, and with an understanding of the basics of swap, you can tweak your system’s use of memory to your heart’s content.

Additional reading

Speed up your system by avoiding the swap file. FOSSWire. Accessed February 8th, 2009.
2.6 swapping behavior. LWN.net. Accessed February 8th, 2009.
Swap FAQ. Ubuntu Documentation. Accessed February 12th, 2009.
Patterson, David A. and Hennessy, John L. Computer Organization and Design: The hardware/software interface. © 1997. Morgan Kaufmann Publishers, San Francisco, California.

¹ As this is only a model, we will naturally be leaving off some of the important but messier details.

² On Linux, you can check your page size with the getconf command, specifying the PAGE_SIZE parameter. This returns the number of pages in bytes. 4 kilobytes (4,096 bytes) is a typical result on the x86 and x86_64architectures, so there are 256 pages / MB.

$ getconf PAGE_SIZE
4096

³ See this Linux kernel mailing list post.

Linux log files

aionman — Wed, 29 Jun 2011 04:29:02 +0000

Linux Log files and usage

=> /var/log/messages : General log messages

=> /var/log/boot : System boot log

=> /var/log/debug : Debugging log messages

=> /var/log/auth.log : User login and authentication logs

=> /var/log/daemon.log : Running services such as squid, ntpd and others log message to this file

=> /var/log/dmesg : Linux kernel ring buffer log

=> /var/log/dpkg.log : All binary package log includes package installation and other information

=> /var/log/faillog : User failed login log file

=> /var/log/kern.log : Kernel log file

=> /var/log/lpr.log : Printer log file

=> /var/log/mail.* : All mail server message log files

=> /var/log/mysql.* : MySQL server log file

=> /var/log/user.log : All userlevel logs

=> /var/log/xorg.0.log : X.org log file

=> /var/log/apache2/* : Apache web server log files directory

=> /var/log/lighttpd/* : Lighttpd web server log files directory

=> /var/log/fsck/* : fsck command log

=> /var/log/apport.log : Application crash report / log file

ssh takes a long time to connect or log in

aionman — Thu, 20 Jan 2011 01:07:24 +0000

Large delays (more than 10 seconds) are typically caused by a problem with name resolution:

Some versions of glibc (notably glibc 2.1 shipped with Red Hat 6.1) can take a long time to resolve “IPv6 or IPv4″ addresses from domain names. This can be worked around with by specifying AddressFamily inet option in ssh_config.
There may be a DNS lookup problem, either at the client or server. You can use the nslookup command to check this on both client and server by looking up the other end’s name and IP address. In addition, on the server look up the name returned by the client’s IP-name lookup. You can disable most of the server-side lookups by setting UseDNS no in sshd_config.

Installation Notes Broadcom tg3 Linux Driver

aionman — Wed, 01 Dec 2010 15:12:28 +0000

Installation Notes
Broadcom tg3 Linux Driver
Version 3.57b
04/28/2006

Broadcom Corporation
16215 Alton Parkway,
Irvine, CA 92619-7013

Table of Contents
=================

Introduction
Limitations
Packaging
Installing Source RPM Package
Building Driver From TAR File
Driver Settings
Driver Defaults
Unloading and Removing Driver
Driver Messages

Introduction
============

This file describes the tg3 Linux driver for the Broadcom NetXtreme
10/100/1000 Mbps PCI/PCI-X/PCI Express Ethernet Network Controllers.
The latest driver is in the latest 2.6 Linux kernel. It can also be
downloaded from http://www.broadcom.com as a source package, but is
generally not necessary to do so if you are using the latest 2.6
upstream kernel from http://www.kernel.org or one of the latest
vendor kernels from Red Hat, SuSE, or others.

The tg3 driver from the Broadcom package is almost identical to the
tg3 driver in the latest 2.6 upstream Linux kernel. It includes some
additional kernel compatible code to allow it to compile on older 2.6
and some 2.4 kernels. The version number is also similar but generally
has a one letter suffix at the end, (e.g. 3.55b) to distinguish it from
the in-kernel tg3 driver.

The next few sections on packaging, compiling, and installation apply
mostly to the Broadcom driver package only.

Limitations
===========

The current version of the driver has been tested on 2.4.x kernels starting
from 2.4.24 and all 2.6.x kernels. The driver may not compile on kernels
older than 2.4.24. Testing is concentrated on i386 and x86_64 architectures.
Only limited testing has been done on some other architectures such as
powerpc and sparc64.

Minor changes to some source files and Makefile may be needed on some
kernels.

Packaging
=========

To replace an older previously installed or in-kernel tg3 driver, follow
the instructions below.

The driver package from http://www.broadcom.com is released in two packaging
formats: source RPM and compressed tar formats. The file names for the two
packages are tg3-.src.rpm and tg3-.tar.gz respectively.
Identical source files to build the driver are included in both packages.

Installing Source RPM Package
=============================

The following are general guidelines for installing the driver.

1. Install the source RPM package:

rpm -ivh tg3-.src.rpm

2. CD to the RPM path and build the binary driver for your kernel:

cd /usr/src/{redhat,OpenLinux,turbo,packages,rpm ..}

rpm -bb SPECS/tg3.spec

rpmbuild -bb SPECS/tg3.spec (for RPM version 4.x.x)

Note that the RPM path is different for different Linux distributions.

3. Install the newly built package (driver and man page):

rpm -ivh RPMS//tg3-..rpm

is the architecture of the machine, e.g. i386:

rpm -ivh RPMS/i386/tg3-.i386.rpm

Note that the –force option may be needed on some Linux distributions
if conflicts are reported.

The driver will be installed in the following path:

2.4.x kernels:

/lib/modules//kernel/drivers/net/tg3.o

2.6.x kernels:

/lib/modules//kernel/drivers/net/tg3.ko

4. Load the driver:

insmod tg3.o
or
insmod tg3.ko (on 2.6.x kernels)
or
modprobe tg3

5. To configure network protocol and address, refer to various Linux
documentations.

Building Driver From TAR File
=============================

The following are general guidelines for installing the driver.

1. Create a directory and extract the files:

tar xvzf tg3-.tar.gz

2. Build the driver tg3.o (or tg3.ko) as a loadable module for the
running kernel:

cd src
make

3. Test the driver by loading it:

insmod tg3.o
or
insmod tg3.ko (on 2.6.x kernels)
or
insmod tg3

4. Install the driver:

make install

See RPM instructions above for the location of the installed driver.

5. To configure network protocol and address, refer to various Linux
documentations.

Driver Settings
===============

This and the rest of the sections below apply to both the in-kernel tg3
driver and the tg3 driver package from Broadcom.

Driver settings can be queried and changed using ethtool. The latest ethtool
can be downloaded from http://sourceforge.net/projects/gkernel if it is not
already installed. The following are some common examples on how to use
ethtool. See the ethtool man page for more information. ethtool settings do
not persist across reboot or module reload. The ethtool commands can be put
in a startup script such as /etc/rc.local to preserve the settings across a
reboot.

1. Show current speed, duplex, and link status:

ethtool eth0

2. Change speed, duplex, autoneg:

Example: 100Mbps half duplex, no autonegotiation:

ethtool -s eth0 speed 100 duplex half autoneg off

Example: Autonegotiation with full advertisement:

ethtool -s eth0 autoneg on

Example: Autonegotiation with 100Mbps full duplex advertisement only:

ethtool -s eth0 speed 100 duplex full autoneg on

3. Show flow control settings:

ethtool -a eth0

4. Change flow control settings:

Example: Turn off flow control

ethtool -A eth0 autoneg off rx off tx off

Example: Turn flow control autonegotiation on with tx and rx advertisement:

ethtool -A eth0 autoneg on rx on tx on

Note that this is only valid if speed is set to autonegotiation.

5. Show offload settings:

ethtool -k eth0

6. Change offload settings:

Example: Turn off TSO (TCP segmentation offload)

ethtool -K eth0 tso off

7. Get statistics:

ethtool -S eth0

8. Perform self-test:

ethtool -t eth0

Note that the interface (eth0) must be up to do all tests.

9. See ethtool man page for more options.

Driver Defaults
===============

Speed : Autonegotiation with all speeds advertised

Flow control : Autonegotiation with rx and tx advertised

MTU : 1500 (range 46 – 9000)

Some chips do not support jumbo MTUs bigger than
1500

Rx Ring Size : 200 (range 0 – 511)

Some chips are fixed at 64

Rx Jumbo Ring Size : 100 (range 0 – 255)

Not all chips support the jumbo ring, and some
chips that support jumbo frames do not use the
jumbo ring.

Tx Ring Size : 511 (range (MAX_SKB_FRAGS+1) – 511)

MAX_SKB_FRAGS varies on different kernels and
different architectures. On a 2.6 kernel for
x86, MAX_SKB_FRAGS is 18.

Coalesce rx usecs : 20 (range 0 – 1023)

Coalesce rx usecs irq : 20 (range 0 – 255)

Coalesce rx frames : 5 (range 0 – 1023)

Coalesce rx frames irq : 5 (range 0 – 255)

Coalesce tx usecs : 72 (range 0 – 1023)

Coalesce tx usecs irq : 20 (range 0 – 255)

Coalesce tx frames : 53 (range 0 – 1023)

Coalesce tx frames irq : 5 (range 0 – 255)

Coalesce stats usecs : 1000000 (aprox. 1 sec.)

Some coalescing parameters are not used or have
different defaults on some chips

MSI : Enabled (if supported by the chip and passed
the interrupt test)

TSO : Enabled on newer chips that support TCP segmentation
offload in hardware.

Unloading and Removing Driver
=============================

To unload the driver, use ifconfig to bring down all eth# interfaces opened
by the driver, then do the following:

rmmod tg3

Note that on 2.6 kernels, it is not necessary to bring down the eth#
interfaces before unloading the driver module.

If the driver was installed using rpm, do the following to remove it:

rpm -e tg3

If the driver was installed using make install from the tar file, the driver
tg3.o (or tg3.ko) has to be manually deleted from the system. Refer
to the section “Installing Source RPM Package” for the location of the
installed driver.

Driver Messages
===============

The following are the most common sample messages that may be logged in the file
/var/log/messages. Use dmesg -n to control the level at which messages
will appear on the console. Most systems are set to level 6 by default. To see
all messages, set the level higher.

Driver signon:
————-

tg3.c:v3.53c (Mar 13, 2006)

NIC detected:
————

eth0: Tigon3 [partno(BCM95704CA40) rev 2002 PHY(5704)] (PCI:66MHz:64-bit) 10/100/1000BaseT Ethernet 00:10:18:04:3e:64
eth0: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0] WireSpeed[1] TSOcap[1]
eth0: dma_rwctrl[763f0000] dma_mask[64-bit]

Link up and speed indication:
—————————-

tg3: eth0: Link is up at 1000 Mbps, full duplex.
tg3: eth0: Flow control is on for TX and on for RX.

Link down indication:
——————–

tg3: eth0: Link is down.

Enable Java Plugin on Firefox 3.6

aionman — Thu, 07 Oct 2010 08:50:27 +0000

Pre-requisites.

I don’t know how much of it its really needed, but I install the whole Sun’s Java 6 packages,

$ sudo apt-get install sun-java6-jdk sun-java-6-jre sun-java6-pluging
   sun-java6-source sun-java6-bin

For sure you’ll need the jdk and plugin one… If your internet connection is slow you might try installing these two first and try the above command.

Configuration of Java for Mozilla.

For enable Java plugin on Firefox 3.6,

$ sudo update-alternatives --install /usr/lib/mozilla/plugins/libjavaplugin.so \
   mozilla-javaplugin.so /usr/lib/jvm/java-6-sun/jre/lib/i386/libnpjp2.so 50

Now it’s time to restart your browser.

For checking that the plugin is enabled, open your Firefox browser and type

about:plugins

in the address line, there must be the Java application enabled.

Install ioncube in ubuntu

aionman — Wed, 28 Apr 2010 02:39:27 +0000

IonCube protects software written using the PHP programming language from being viewed, changed, and run on unlicensed computers.

1. Download ionCube loaders

sudo wget http://downloads.ioncube.com/loader_downloads/ioncube_loaders_lin_x86.tar.gz

2. Extract

sudo tar zxvf ioncube_loaders_lin_x86.tar.gz

3. Move to a permanent location

sudo mv ioncube /usr/local/

4. Add reference to your php.ini file (sudo pico /etc/php5/apache2/php.ini)

zend_extension = /usr/local/ioncube/ioncube_loader_lin_5.2.so

There are a few versions of the loader in the tar archive. Use the one that matches your PHP version.

5. Restart apache

sudo /etc/init.d/apache2 restart

6. check installation using following command:
php -v
you should see following output:

PHP 5.1.6 (cli) (built: Apr 7 2009 08:00:04)
Copyright (c) 1997-2006 The PHP Group
Zend Engine v2.1.0, Copyright (c) 1998-2006 Zend Technologies
with the ionCube PHP Loader v3.1.34, Copyright (c) 2002-2009, by ionCube Ltd.

VMWare – Guest cannot browse Internet

aionman — Sun, 11 Apr 2010 01:13:16 +0000

I think I figured it out. It isn’t related to SLlinux, nor to port 443, nor to VMWare and the FC5 port directly, but a combination of the Fedora port and VMware settings. It is related to the MTU setting consistancy between the virtual machine, FC5 and the site in question. For some reason the FC5 was set to 1492 and not 1500 as set under FC3. For this reason some of the HTTPS sites, I assume, require 1500 mtu. All I did was set Fedora’s eth0′s MTU setting to 1500 (ip link set eth0 mtu 1500) and it works fine. Weird. I could be wrong, but I think that was the problem. At least now it works. I got this hint from the guys at Fidelity. They thought it was unrelated, but apparently it is.

Maximum Transmission Unit(MTU), the largest physical packet size, measured in bytes, that a network can transmit. Any messages larger than the MTU are divided into smaller packets before being sent.By optimizing the MTU setting you can gain substantial network performance increases, especially when using dial-up modem connections.

Default MTU Size for Different Network Topology

Network	MTU(Bytes)
16 Mbit/Sec Token Ring	17914
4 Mbits/Sec Token Ring	4464
FDDI	4352
Ethernet	1500
IEEE 802.3/802.2	1492
X.25	576

To change the MTU of an interface on GNU/Linux, you just need to use ifconfig command to do so, like this for example

sudo ifconfig eth0 mtu 1492

To change it permanently on Debian, put it in the /etc/network/interfaces file .where almost all network parameters are found. To do this, just add a line mtu to the definition of your interface and save the file.

sudo gedit /etc/network/interfaces

Example

iface eth0 inet static
address 192.168.0.1
network 192.168.0.0
gateway 192.168.0.254
netmask 255.255.255.0
mtu 1492

Daiup Users

For dialup users: the Maximum Transmission Unit (MTU) value can be changed within the file
/etc/ppp/options

Replacing A Failed Hard Drive In A Software RAID1/5 Array

aionman — Thu, 12 Nov 2009 01:05:20 +0000

Replacing A Failed Hard Drive In A Software RAID1 Array

Version 1.0
Author: Falko Timme
Last edited 01/21/2007

This guide shows how to remove a failed hard drive from a Linux RAID1/5 array (software RAID), and how to add a new hard disk to the RAID1/5 array without losing data.

Take RAID1 as an example.

1 Preliminary Note

I have two hard drives, /dev/sda and /dev/sdb, with the partitions /dev/sda1 and /dev/sda2 as well as /dev/sdb1 and /dev/sdb2.

/dev/sda1 and /dev/sdb1 make up the RAID1 array /dev/md0.

/dev/sda2 and /dev/sdb2 make up the RAID1 array /dev/md1.

/dev/sda1 + /dev/sdb1 = /dev/md0

/dev/sda2 + /dev/sdb2 = /dev/md1

/dev/sdb has failed, and we want to replace it.

2 How Do I Tell If A Hard Disk Has Failed?

If a disk has failed, you will probably find a lot of error messages in the log files, e.g. /var/log/messages or /var/log/syslog.

You can also run

cat /proc/mdstat

and instead of the string [UU] you will see [U_] if you have a degraded RAID1 array.

3 Removing The Failed Disk

To remove /dev/sdb, we will mark /dev/sdb1 and /dev/sdb2 as failed and remove them from their respective RAID arrays (/dev/md0 and /dev/md1).

First we mark /dev/sdb1 as failed:

mdadm –manage /dev/md0 –fail /dev/sdb1

The output of

cat /proc/mdstat

should look like this:

server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid5] [raid4] [raid6] [raid10]
md0 : active raid1 sda1[0] sdb1[2](F)
24418688 blocks [2/1] [U_]

md1 : active raid1 sda2[0] sdb2[1]
24418688 blocks [2/2] [UU]

unused devices:

Then we remove /dev/sdb1 from /dev/md0:

mdadm –manage /dev/md0 –remove /dev/sdb1

The output should be like this:

server1:~# mdadm –manage /dev/md0 –remove /dev/sdb1
mdadm: hot removed /dev/sdb1

And

cat /proc/mdstat

should show this:

server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid5] [raid4] [raid6] [raid10]
md0 : active raid1 sda1[0]
24418688 blocks [2/1] [U_]

md1 : active raid1 sda2[0] sdb2[1]
24418688 blocks [2/2] [UU]

unused devices:

Now we do the same steps again for /dev/sdb2 (which is part of /dev/md1):

mdadm –manage /dev/md1 –fail /dev/sdb2

cat /proc/mdstat

mdadm –manage /dev/md1 –remove /dev/sdb2

server1:~# mdadm –manage /dev/md1 –remove /dev/sdb2
mdadm: hot removed /dev/sdb2

cat /proc/mdstat

Then power down the system:

shutdown -h now

and replace the old /dev/sdb hard drive with a new one (it must have at least the same size as the old one – if it’s only a few MB smaller than the old one then rebuilding the arrays will fail).

4 Adding The New Hard Disk

After you have changed the hard disk /dev/sdb, boot the system.

The first thing we must do now is to create the exact same partitioning as on /dev/sda. We can do this with one simple command:

sfdisk -d /dev/sda | sfdisk /dev/sdb

You can run

fdisk -l

to check if both hard drives have the same partitioning now.

Next we add /dev/sdb1 to /dev/md0 and /dev/sdb2 to /dev/md1:

mdadm –manage /dev/md0 –add /dev/sdb1

server1:~# mdadm –manage /dev/md0 –add /dev/sdb1
mdadm: re-added /dev/sdb1

mdadm –manage /dev/md1 –add /dev/sdb2

server1:~# mdadm –manage /dev/md1 –add /dev/sdb2
mdadm: re-added /dev/sdb2

Now both arays (/dev/md0 and /dev/md1) will be synchronized. Run

cat /proc/mdstat

to see when it’s finished.

During the synchronization the output will look like this:

server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid5] [raid4] [raid6] [raid10]
md0 : active raid1 sda1[0] sdb1[1]
24418688 blocks [2/1] [U_]
[=>...................] recovery = 9.9% (2423168/24418688) finish=2.8min speed=127535K/sec

md1 : active raid1 sda2[0] sdb2[1]
24418688 blocks [2/1] [U_]
[=>...................] recovery = 6.4% (1572096/24418688) finish=1.9min speed=196512K/sec

unused devices:

When the synchronization is finished, the output will look like this:

server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid5] [raid4] [raid6] [raid10]
md0 : active raid1 sda1[0] sdb1[1]
24418688 blocks [2/2] [UU]

md1 : active raid1 sda2[0] sdb2[1]
24418688 blocks [2/2] [UU]

unused devices:

That’s it, you have successfully replaced /dev/sdb!

How To Install VMware Server On A CentOS 5

aionman — Fri, 14 Aug 2009 03:12:49 +0000

How To Install VMware Server On A CentOS 5.0 Desktop

This tutorial provides step-by-step instructions on how to install VMware Server on a CentOS 5.0 desktop system. With VMware Server you can create and run guest operating systems (“virtual machines”) such as Linux, Windows, FreeBSD, etc. under a host operating system. This has the benefit that you can run multiple operating systems on the same hardware which saves a lot of money, and you can move virtual machines from one VMware Server to the next one (or to a system that has the VMware Player which is also free).

Also, with VMware Server you can let your old Windows desktop (that you previously converted into a VMware virtual machine with VMware Converter, as described in this tutorial:http://www.howtoforge.com/vmware_converter_windows_linux) run under your CentOS desktop. This can be useful if you depend on some applications that exist for Windows only, or if you want to switch to Linux slowly.

I want to say first that this is not the only way of setting up such a system. There are many ways of achieving this goal but this is the way I take. I do not issue any guarantee that this will work for you!

1 Find Out Your Kernel Version

Before we go on and install additional software, it’s a good idea to find out about your kernel version because in chapter 2 we will install the package kernel-devel which is needed by VMware Server. There are multiple kernel-devel packages available, and to select the right one you need to know your kernel version.

To find out about your kernel version, open a terminal (Applications > Accessories > Terminal):

Then become root by running:

Then run

uname -r

The output should look like this:

[root@localhost Desktop]# uname -r
2.6.18-8.1.3.el5

which means you have kernel 2.6.18-8.1.3.el5 installled.

2 Installing Required Packages

Before we install VMware Server, we must install some prerequisites. To install them, go to Applications > Add/Remove Software:

Type in the root password:

The Package Manager opens. Go to the Browse tab and select:

Development > Development Libraries
Development > Development Tools

Then go to the Search tab and search for xinetd. Select the xinetd package for installation:

Do the same for the kernel-devel package. Please make sure you select the kernel-devel package that corresponds to your current kernel (so if you kernel is 2.6.18-8.1.3.el5, select the kernel-devel – 2.6.18-8.1.3.el5.i686 package).

Click on Apply afterwards.

The Package Manager will then resolve all dependencies, download the packages, maybe ask you to accept some unknown software keys (please accept them), and finally install the packages.

3 VMware Server

To download VMware Server, go to http://www.vmware.com/download/server/ and click on Download Now:

Accept the license agreement by clicking on Yes:

Then download the VMware Server for Linux .tar.gz file (not the rpm file!) to your desktop (e.g. to /home/falko/Desktop):

To get the serial number you need to run VMware Server, go to http://register.vmware.com/content/registration.html. Fill in your personal details. Afterwards you will get a page with a serial number for VMware Server. Write it down or print it out:

To install VMware Server, open a terminal (Applications > Accessories > Terminal) and become root:

Then go to the location where you saved the VMware Server .tar.gz file, e.g. /home/falko/Desktop (replace falko with your own username!):

cd /home/falko/Desktop

Unpack the VMware Server .tar.gz file and run the installer:

tar xvfz VMware-server-*.tar.gz
cd vmware-server-distrib
./vmware-install.pl

The installer will ask you a lot of questions. You can always accept the default values simply by hitting . When it asks you

In which directory do you want to keep your virtual machine files?
[/var/lib/vmware/Virtual Machines]

you can accept the default value or specify a different location where you have more free disk space, e.g. like /home/falko/virtual_machines, but this is up to you and not necessary.

At the end of the installation, you will be asked to enter a serial number:

Please enter your 20-character serial number.

Type XXXXX-XXXXX-XXXXX-XXXXX or ‘Enter’ to cancel:

Fill in your serial number for VMware Server.

After the successful installation, you can delete the VMware Server download file and the installation directory:

cd ../
rm -f VMware-server*
rm -fr vmware-server-distrib/

You will now find VMware Server under Applications > System Tools:

When you start it, select Local host:

With VMware 2.0 The most noticeable change is that the vmware server console is … gone.

Afterwards, you can create virtual machines (or import your virtual Windows machine that you created with VMware Converter):

Replacing A Failed Hard Drive In A Software RAID1 Array

aionman — Tue, 04 Aug 2009 05:22:09 +0000

This guide shows how to remove a failed hard drive from a Linux RAID1 array (software RAID), and how to add a new hard disk to the RAID1 array without losing data.

I do not issue any guarantee that this will work for you!

1 Preliminary Note

In this example I have two hard drives, /dev/sda and /dev/sdb, with the partitions /dev/sda1 and /dev/sda2 as well as /dev/sdb1 and /dev/sdb2.

/dev/sda1 and /dev/sdb1 make up the RAID1 array /dev/md0.

/dev/sda2 and /dev/sdb2 make up the RAID1 array /dev/md1.

/dev/sda1 + /dev/sdb1 = /dev/md0

/dev/sda2 + /dev/sdb2 = /dev/md1

/dev/sdb has failed, and we want to replace it.

2 How Do I Tell If A Hard Disk Has Failed?

If a disk has failed, you will probably find a lot of error messages in the log files, e.g. /var/log/messages or /var/log/syslog.

You can also run

cat /proc/mdstat

and instead of the string [UU] you will see [U_] if you have a degraded RAID1 array.

3 Removing The Failed Disk

To remove /dev/sdb, we will mark /dev/sdb1 and /dev/sdb2 as failed and remove them from their respective RAID arrays (/dev/md0 and /dev/md1).

First we mark /dev/sdb1 as failed:

mdadm –manage /dev/md0 –fail /dev/sdb1

The output of

cat /proc/mdstat

should look like this:

server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid5] [raid4] [raid6] [raid10]
md0 : active raid1 sda1[0] sdb1[2](F)
24418688 blocks [2/1] [U_]

md1 : active raid1 sda2[0] sdb2[1]
24418688 blocks [2/2] [UU]

unused devices:

Then we remove /dev/sdb1 from /dev/md0:

mdadm –manage /dev/md0 –remove /dev/sdb1

The output should be like this:

server1:~# mdadm –manage /dev/md0 –remove /dev/sdb1
mdadm: hot removed /dev/sdb1

And

cat /proc/mdstat

should show this:

server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid5] [raid4] [raid6] [raid10]
md0 : active raid1 sda1[0]
24418688 blocks [2/1] [U_]

md1 : active raid1 sda2[0] sdb2[1]
24418688 blocks [2/2] [UU]

unused devices:

Now we do the same steps again for /dev/sdb2 (which is part of /dev/md1):

mdadm –manage /dev/md1 –fail /dev/sdb2

cat /proc/mdstat

server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid5] [raid4] [raid6] [raid10]
md0 : active raid1 sda1[0]
24418688 blocks [2/1] [U_]

md1 : active raid1 sda2[0] sdb2[2](F)
24418688 blocks [2/1] [U_]

unused devices:

mdadm –manage /dev/md1 –remove /dev/sdb2

server1:~# mdadm –manage /dev/md1 –remove /dev/sdb2
mdadm: hot removed /dev/sdb2

cat /proc/mdstat

server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid5] [raid4] [raid6] [raid10]
md0 : active raid1 sda1[0]
24418688 blocks [2/1] [U_]

md1 : active raid1 sda2[0]
24418688 blocks [2/1] [U_]

unused devices:

Then power down the system:

shutdown -h now

and replace the old /dev/sdb hard drive with a new one (it must have at least the same size as the old one – if it’s only a few MB smaller than the old one then rebuilding the arrays will fail).

4 Adding The New Hard Disk

After you have changed the hard disk /dev/sdb, boot the system.

The first thing we must do now is to create the exact same partitioning as on /dev/sda. We can do this with one simple command:

sfdisk -d /dev/sda | sfdisk /dev/sdb

You can run

fdisk -l

to check if both hard drives have the same partitioning now.

Next we add /dev/sdb1 to /dev/md0 and /dev/sdb2 to /dev/md1:

mdadm –manage /dev/md0 –add /dev/sdb1

server1:~# mdadm –manage /dev/md0 –add /dev/sdb1
mdadm: re-added /dev/sdb1

mdadm –manage /dev/md1 –add /dev/sdb2

server1:~# mdadm –manage /dev/md1 –add /dev/sdb2
mdadm: re-added /dev/sdb2

Now both arays (/dev/md0 and /dev/md1) will be synchronized. Run

cat /proc/mdstat

to see when it’s finished.

During the synchronization the output will look like this:

md1 : active raid1 sda2[0] sdb2[1]
24418688 blocks [2/1] [U_]
[=>...................] recovery = 6.4% (1572096/24418688) finish=1.9min speed=196512K/sec

unused devices:

When the synchronization is finished, the output will look like this:

server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid5] [raid4] [raid6] [raid10]
md0 : active raid1 sda1[0] sdb1[1]
24418688 blocks [2/2] [UU]

md1 : active raid1 sda2[0] sdb2[1]
24418688 blocks [2/2] [UU]

unused devices:

That’s it, you have successfully replaced /dev/sdb!