[Link] TCP/IP의 이해(Protocol Suite + IP Address & Subnet)

[출처] Secure.pe.kr 강의실

3.TCP/IP의 이해
    1) TCP/IP Protocol Suite

1.TCP/IP Protocol의 개요

2.TCP/IP Protocol Suite

3.TCP/IP Sliding Window

4. Windows Size의 크기에 따른 영향

5. IP Packet의 기본흐름의 이해

    2) IP Address

1. IP Address의 이해

2. IP Addressing

3. Subnet Mask, Default Gateway

4. CIDR (Subnetting,Supernetting), VLSM

[Scrap] 리눅스 서버의 보안설정

리눅스 서버의 보안설정 모음(I)      
일반적인 시스템 보안 설정 및 최적화

리눅스베이 (master@ilinuxbay.com)     |   2003.08.10

▮ root 접속시간 설정
/etc/profile에 "TMOUT=300" 으로 초단위 접속 시간을 설정한다.
설정 시간 동안 아무 작업도 하지 않으면 자동 로그아웃 된다.

▮ /etc/securetty
이 파일은 root가 로그인할 수 있는 터미널 장치(TTY)와 가상 콘솔(VC)을 정의 한다.
login 프로그램은 이 파일을 참조하여 프로세스를 처리한다.
사용하지 않을 장치에 한하여 삭제하거나 주석 처리한다. 그러면 root는 해당 장치로 로그인 할 수 없을 것이다. 추천사항으로 하나의 vc/1과 tty1만을 허용하는 것을 추천한다.

▮ Ctrl-Alt-Del 키를 사용한 부팅 막기
/etc/inittab 파일의 다음 부분을 주석 처리한다
ca::ctrlaltdel:/sbin/shutdown -t3 -r now

▮ /etc/service 파일
[root@ezo]# chattr +i /etc/service

▮ shell 설정
/etc/profile에서 시스템의 모든 사용자에 대해 HISTSIZE를 설정한다.
이는 .bash_profile 파일이 가질 수 있는 명령어수를 지정한다.
HISTSIZE=10
다음과 같이 설정하여 침입자가 현재 시스템에 로그인 되어 있지 않은 사용자의 .bash_profile 파일을 사용하지 못하게 한다.
HISTFILESIZE=0

▮ 파일 시스템 마운트 제어
/etc/fstab에 몇가지 옵션을 주어 설정하고 제어 할 수 있다.
일단 LABEL=/boot와 같은 이름을 /dev/hda1와 같이 파티션 장치명으로 바꾸어 준다.
이는 관리차원과 쿼터를 사용하게 될 때 문제가 될 수 있는 부분을 없애기 위함이다.
/boot 파티션 읽기 전용으로 설정
/dev/hda1 /boot ext3 default,ro 1 2
적용하려면
mount /boot -oremount
확인
cat /proc/mount

▮ chmod로 파일의 권한 변경
아래의 파일들은 침입자가 악용할 수 있는 파일들 이므로 root이외의 권한은 제거한다.
이외에도 설치한 패키지에 따라 권한을 제거해야 할 파일들이 있다.
wget, finger, nslookup그리고 gcc같은 컴파일러 패키지들은 설치 여부에 따라 존재하지 않을 수도 있다.
예] chmod 700 /usr/bin/whereis /usr/bin/consolehelper /usr/bin/pstree
/usr/bin/which /usr/bin/who /usr/bin/w /usr/bin/top
/usr/bin/find /bin/mail /bin/ps /usr/bin/wget /usr/bin/finger
/usr/bin/nslookup /usr/bin/gcc /usr/bin/cc
/usr/bin/c++ /usr/bin/make

[Link] TCP/IP Network Basics / Cisco Routing

Daum 신지식 Link (Original source : www.securitylab.co.kr)

1. TCP/IP Network 개념

2. Cisco Routing

[Info] CCNA

CCNA 시험유형은 다지선다형 객관식과 Drag & Drop, 단답형 주관식, Simulation(2문제)와 주관식(2문제) 형식으로 나온다.
시험문제는 50~65문제로 1000점 만점에 849점을 맞으면 합격. 유효기간은 3년.

2007년 8월부터 NEW CCNA 시험
640-801 시험이 640-802 시험으로, 640-821 + 640-811 시험이 640-822 + 640-816 시험으로 바뀌었다. 기존 시험은 2007년 11월 6일 까지 응시가 가능하다.
새로운 640-802 CCNA 시험이나 [640-822 ICND1 + 640-816 ICND2] 시험은 2007년 8월 15일부터 응시가 가능.

2007년 8월 15일 ~ 11월 6일 까지는 신/구 시험 모두 응시가 가능하며 11월 6일 이후에는 640-802 CCNA 또는  640-822 ICND1 + 640-816 ICND2 로만 시험응시 가능.

CCNA 취득을 위한 실습공부

Boson Simulation S/W로 연습
Simulation에서 각 Mode에 대한 개념을 확실히 파악
Privilege Mode, Router Mode, Interface Mode 등을 이해
명령어 암기
Protocol을 이해하는 것이 가장 중요.

Wikipedia에 소개된 정보

Cisco Certified Networking Associate (CCNA)

Candidates have the option of gaining the certification by passing two tests (INTRO and ICND), or one single test (CCNA); the two-test option has the advantage of allowing the candidate to focus on certain subjects.

The certification is valid for three years; at that time a CCNA holder must either

• re-take the CCNA or ICND exam, or
• take and pass an exam for one of the Professional (e.g., CCNP) or Specialist level certifications (excluding the sales specialist exams), or pass the CCIE written exam.

These exams are known by their corresponding numbers. When the curriculum of the exam changes the exam number also changes. The current exam number for CCNA is 640-802 (from 15 Aug 2007). The exam number for INTRO is 640-821 (Last day to test 11/6/07) and ICND the exam number is 640-811 (Last day to test 11/6/07). New ICND Part1 (640-822 ICND1) and ICND Part2 (640-816 ICND2) available from 15 Aug 2007.

These exams are conducted by authorized test centers at a cost of $100 for the INTRO or ICND exam and $150 for the full CCNA exam.

Cisco의 CCNA FAQs에서 발췌

Q: When should a student take the Cisco Certified Network Associate exam?

A: Cisco recommends taking the CCNA exam after completing the fourth course of the CCNA curriculum.

Q: Where is the CCNA test given?

A: The CCNA test is given at Pearson VUE testing centers worldwide.

[scrap] 시스템 관리자가 되려면

출처 : 코리아인터넷닷컴, a 2001년 02월 12일 저자 : d_i_d 시스템 관리자가 되려면 (1) : 시스템 관리자의 자세

필자는 시스템 관리자로 일하고 있다. 이 분야에서 먼저 일한 사람으로서 시스템 관리자가 되고 싶은 많은 사람들에게 몇 가지 조언을 하고 싶다. 시스템 관리자의 법칙 : 1. 모든 것을 알아야 한다. 2. 만일 모든 것을 알 수 없다면, 필요한 정보를 어디서 얻을 수 있는지 알아야 한다. - 필자는 항상 많은 책들을 손 닿기 쉬운 곳에 두며 월급날마다 새로운 책을 산다. 이제부터라도 자신만의 도서관을 만들기 시작해야 한다. 그렇게 모으는 책들의 일부는 시간이 지남에 따라 소용없는 것이 되기도 하겠지만, 대부분의 책들은 그렇지 않다. 3. 필요한 정보를 찾을 수 없다면, 해답을 찾기 위해서 기꺼이 시간을 투자하라. 4. 어떤 것이 되었든 간에 항상 기술 관련 서적을 가지고 다니도록 하라!! -차를 타고 가는 도중 꽉 막혀 있으면, 가지고 있는 책을 읽도록 하라. 화장실에 앉아 있을 때도 책을 읽어라. 필자는 하루에 적어도 3시간 동안은 기술에 관련된 글을 읽는다. 아무리 많이 읽어도 충분히 읽었다고 할 수 없다. 5. 여러분이 모르는 것을 알고 있는 사람들과 친분 관계를 쌓도록 하라. - 필자에게 필요하지는 않지만 다른 친구들에게 필요한 해결책을 게시판에 올리는 경우가 많다. 서버 관련된 내용을 다루는 웹사이트를 통하면 친구들을 사귈 수 있다.

시스템 관리자가 되려면 (2) : 반드시 갖추어야 할 기술과 지식

1. TCP/IP와 관련된 서비스들과 프로그램들. DNS, SMTP, UDP, HTTP, FTP 등이 여기에 포함된다. 이들에 대해서 모든 것을 알아야 할 필요는 없지만 필요한 정보를 어디서 얻을 수 있는지는 알아야 한다. 시스템 관리자의 법칙 2번째를 보도록 하라. -시스템 관리자는 TCP/IP가 실행되는 컴퓨터를 다루기 때문에 이에 대한 지식은 매우 중요하다. 다른 시스템 관리자의 일을 인수하는 것도 나쁘지는 않지만, 만일 누군가 추가적인 서비스를 요구한다면 TCP/IP에 대한 지식이 반드시 필요하다. 2. 아마도 가장 중요한 사항이 될 것인데, 가능한 많은 운영 체제에 대해서 알아야 한다. Solaris, Linux, IRIX, HP UX를 포함하는 것은 물론, Windows까지도 알아야 한다. -이렇게 함으로써 여러분의 몸값을 높일 수 있다. 또한 여러 가지 운영 체제에 대해서 알게 되면, 하고자 하는 작업에 가장 알맞은 운영 체제를 고른다든지 할 때 보다 많은 해결 방안을 내놓을 수 있게 된다. 리눅스가 항상 최상의 해답은 아니다. 모든 것은 그것 자체의 작은 흠을 가지고 있다. Solaris는 리눅스와 약간은 다르게 작동하며, HP-UX 등도 그렇다. 가능한 많은 것을 알려고 노력하고 위의 황금률 2번째를 명심하라. 3. 네트워킹, 개념과 하드웨어 - 이것까지 알아야 하는 이유는, 중소 기업에서 일해야 할 경우, 여러분에게 네트워크에 대한 해결까지 기대할 것이기 때문이다. 필자는 라우터에 대해 잘 알지 못하는데도, 적어도 내가 원하는 대로 작동하도록 라우터를 설정할 수는 있다. 필자는 스위치, 패치 패널, 방화벽(하드웨어와 소프트웨어 모두)를 운용할 수 있다. 혹시 전문가 아니냐고? 나는 전혀 전문가가 아니다! 하지만 적어도 네트워크에 관련된 것으로 내가 해야 하는 일이라면 파고 들 수 있다. 여러분이 지원하고 있는 많은 사람들은 여러분이 모든 것을 알고 있기를 기대한다. 그들의 기대를 저버리지 않도록 노력하라. 4. 백업 시스템과 테크닉 - 나도 안다. 이것이 얼마나 지루한가에 대해서는 나도 동의한다. 하지만 시스템 관리자의 업무에서 매우 중요한 부분이다. 백업하는 방법, 가장 좋은 백업 방법 등에 관한 것을 알아 놓도록 하라. 5. 문서 작성 능력 - 유별나게 뛰어난 문서 작성 능력을 요구하는 것이 아니다. 그저 잘 할 수 있으면 된다. 시스템 관리자로서 하는 모든 작업은 문서화될 필요가 있다. 언제 어디서 무슨 일이 일어날 지 모르기 때문이다. 만일 내게 문제가 생긴다면, 다른 시스템 관리자가 들어와서 내가 했던 일을 배워야 할 것이다. 또한 여러분의 상사와 관리자들은 이렇게 문서화하는 것을 무척이나 좋아한다. 정작 문서화된 것의 내용은 전혀 알지도 못하면서도 말이다! 6. 부당한 처사에도 관대히 용서할 수 있는 태도 -이것은 시스템 사용자를 대할 때 필요한 것이다. 사용자들은 이런 태도를 전혀 가지고 있지 않다고 생각하면 맞을 것이다. 여러분 또는 여러분의 상사는 이런 태도를 항상 가지고 있어야 할 것이다. 뭔가 잘못되었을 때, 모든 비난의 화살은 여러분을 향하게 된다. 이런 사실을 알고 감수하도록 하라. 7. 컴퓨터 하드웨어 -SCSI 및 IDE/EIDE 등의 하드 디스크를 추가하는 방법을 알도록 하라. 다시 말하지만, 여러분이 이런 모든 일을 알고 있다고 다른 사람들이 기대할 것이기 때문이다. 8. 스크립팅 언어 -필자는 bash와 perl을 알고 있다. 리눅스에서는 bash가 쓰이고, 또 필자가 다루고 있는 많은 기계가 리눅스이기 때문이며, perl은 이식성이 매우 높기 때문이다. 사실, perl이 매우 뛰어나기 때문에 bash 스크립트를 작성하는 대신에 perl로 작성해서 마무리하는 경우가 많다. 9. 보안 문제 -어떤 서비스와 프로그램들이 취약한지, 그렇다면 취약성을 보완하기 위해서 어떻게 바꾸고 설정하며 패치를 할 것인지 알아야 한다.

[Info] [Link] Cisco CCNA Test : CCNA Training for the CCNA Test

CramSession

Cisco CCNA Test: CCNA - Training for the CCNA Test

블로그 방문 5000회 돌파

오늘 부로 방문자수가 5000을 넘어섰다.
블로그 개설 후 8개월만이다.
3개월 째부터 일정 수준의 방문객 수가 유지되면서 완만한 상승곡선을 이루긴 했었지만
이번 9월 방문자수가 2500이 넘어 8월에 비해 1800이 늘어나는 놀라운 성장이 있었다.
이건 나로서는 전혀 예상치 못했던 일로서
결정적인 원인은 Linux APM Server에 관한 몇 개의 글을 올린데에 있었던 것 같다.
일단은 방문해주신 모든 분들께 이자리를 빌어 감사의 말씀을 드린다.

스킨편집의 고행길

[Talk 1]
원래 블로그에 나와 다른 사람들에게 유용한 글을 올리는 것 이외의 작업을 위해 많은 시간을 할애하는 일은 피하려고 하였다.
하지만 사람 일이 꼭 마음먹은 바대로 되는 것은 아닌지라 근래에 스킨에 손을 대는 시간이 많아지고 있다.
애초에 애드센스나 CSE를 사용하기 위해서는 스킨에 손을 대지 않을 수 없는 것이었지만 결국 이 사소한 것들을 핑계로 수정의 범위가 블로그 전체로 늘어나버렸다.
모든 점이 마음에 쏙드는 스킨을 좀처럼 찾을 수가 없었기에 처음에는 단순히 기존에 있는 스킨들 중에서 마음에 드는 부분들만을 모아 쓰려고 했는데 상당히 많은 시행착오를 겪게 되어버렸다.
나는 css가 널리 사용되기 훨씬 이전에 html을 공부했고, 그 후에는 그다지 그쪽으로 공부를 하지 않아서 css의 개념 정도밖에 알고 있지 못했다. 그래서 내가 원하는 모양의 스킨을 만들어 내기가 쉽지 않았다.
또한 처음에 기반으로 삼으려던 스킨이 상당히 오래된 것이어서 최신의 스킨들과 코드사용구조가 많이 다른 것이 또하나의 어려움이었다.
삽질을 좀 하다보니 이제서야 좀 구체적인 방안이 잡히긴 했지만 실행에 옮기려니 심히 번거롭다 하지 않을 수가 없다.
잠깐 한숨을 돌리고 머리가 좀 식은 후에 작업을 재개하도록 해야겠다.

[Talk 2]
상당히 많은 부분을 고치고 나니 이젠 어느 정도 원하던 모습에 가까워지고 있다.
워낙 많이 보다보니 skin.html과 style.css의 구조는 거의 이해가 되었다.
그렇지만 계속해서 문제점이 새로이 발견되고 있다.
Google CSE는 다시 삽입하였는데 결과창이 너무 작게 나타나는 문제가 있었다.
우선은 googleSearchFrameWidth 값을 찾아서 content창의 넓이에 맞게 고치고 나니 일단은 문제가 해결되었다.

[Source] IBM developerWorks Tuning LAMP systems, Part 2

Tuning LAMP systems, Part 2: Optimizing Apache and PHP

What slows Apache down, and how to get the most out of PHP

Level: Intermediate

Sean A. Walberg (sean@ertw.com), Senior Network Engineer

30 Apr 2007

Applications using the LAMP (Linux®, Apache, MySQL, PHP/Perl) architecture are constantly being developed and deployed. But often the server administrator has little control over the application itself because it's written by someone else. This series of three articles discusses many of the server configuration items that can make or break an application's performance. This second article focuses on steps you can take to optimize Apache and PHP.

Linux, Apache, MySQL, and PHP (or Perl) form the basis of the LAMP architecture for Web applications. Many open source packages based on LAMP components are available to solve a variety of problems. As the load on an application increases, the bottlenecks in the underlying infrastructure become more apparent in the form of slow response to user requests. The previous article showed you how to tune the Linux system and covered the basics of LAMP and performance measurement. This article focuses on the Web server components, Apache and PHP.

Tuning Apache

Apache is a highly configurable piece of software. It has a lot of features, but each one comes at a price. Tuning Apache is partially an exercise in proper allocation of resources, and involves stripping down the configuration to only what's needed.

Configuring the MPM

Apache is modular in that you can add and remove features easily. Multi-Processing Modules (MPMs) provide this modular functionality at the core of Apache -- managing the network connections and dispatching the requests. MPMs let you use threads or even move Apache to a different operating system.

Only one MPM can be active at one time, and it must be compiled in statically with --with-mpm=(worker|prefork|event) .

The traditional model of one process per request is called prefork. A newer, threaded, model is called worker, which uses multiple processes, each with multiple threads to get better performance with lower overhead. The final, event MPM is an experimental module that keeps separate pools of threads for different tasks. To determine which MPM you're currently using, execute httpd -l.

Choosing the MPM to use depends on many factors. Setting aside the event MPM until it leaves experimental status, it's a choice between threads or no threads. On the surface, threading sounds better than forking, if all the underlying modules are thread safe, including all the libraries used by PHP. Prefork is the safer choice; you should do careful testing if you choose worker. The performance gains also depend on the libraries that come with your distribution and your hardware.

Regardless of which MPM you choose, you must configure it appropriately. In general, configuring an MPM involves telling Apache how to control how many workers are running, whether they're threads or processes. The important configuration options for the prefork MPM are shown in Listing 1.

Listing 1. Configuration of the prefork MPM

                
StartServers       50
MinSpareServers   15
MaxSpareServers   30
MaxClients       225
MaxRequestsPerChild  4000

In the prefork model, a new process is created per request. Spare processes are kept idle to handle incoming requests, which reduces the start-up latency. The previous configuration starts 50 processes as soon as the Web server comes up and tries to keep between 10 and 20 idle servers running. The hard limit on processes is dictated by MaxClients. Even though a process can handle many consecutive requests, Apache kills off processes after 4,000 connections, which mitigates the risk of memory leaks.

Configuring the threaded MPMs is similar, except that you must determine how many threads and processes are to be used. The Apache documentation explains all the parameters and calculations necessary.

Choosing the values to use involves some trial and error. The most important value is MaxClients. The goal is to allow enough worker processes or threads to run without causing your server to swap excessively. If more requests come in than can be handled, then at least those that made it through get service; the others are blocked.

If MaxClients is too high, then all clients experience poor service because the Web server tries to swap out one process to allow another one to run. Too low a setting means you may deny services unnecessarily. Checking the number of processes running at high loads and the resulting memory footprint of all the Apache processes gives you a good idea of how to set this value. If you go over 256 MaxClients, you must also set ServerLimit to the same number; read the MPM's documentation carefully for the associated caveats.

Tuning the number of servers to start and keep spare depends on the role of the server. If the server runs only Apache, you can use modest values as shown in Listing 1, because you're able to make full use of the machine. If the system is shared with a database or other server, then you should limit the number of spare servers being run.

Compiling your own software

When I got started with UNIX®, I insisted on compiling my own software for everything I put on my systems. Maintaining updates eventually caught up with me, so I learned how to build packages to ease this task. Eventually, I realized that most of the time I was duplicating the effort the distribution was doing; now, for the most part, I stick with whatever is provided by my distribution of choice when I can, and roll my own packages when I must.

Similarly, you may find that maintainability of vendor packages outweighs the benefits of going with the latest and greatest code. Sometimes performance tuning and systems administration have conflicting goals. You may have to consider vendor support if you're using a commercial Linux or relying on third-party support.

If you strike out on your own, learn how to build packages that work with your distribution and how to integrate them into your patching system. This will ensure that the software, along with any tweaks you make, are built consistently and can be used across multiple systems. Also keep on top of software updates by subscribing to the appropriate mailing lists and Rich Site Summary (RSS) feeds.

Using options and overrides efficiently

Each request that Apache processes goes through a complicated set of rules that dictates any restrictions or special instructions the Web server must follow. Access to a folder can be restricted by IP address to a certain folder, or a username and password can be configured. These options also include the handling of certain files, such as if a directory listing is provided, how certain filetypes are to be handled, or whether the output should be compressed.

These configurations take the form of containers in httpd.conf such as <Directory> to specify that the configuration to follow refers to a location on disk, or <Location> to indicate that the reference is to a path in the URL. Listing 2 shows a Directory container in action.

Listing 2. A Directory container being applied to the root directory

<Directory /> AllowOverride None Options FollowSymLinks </Directory>

In Listing 2, the configuration enclosed in the Directory and /Directory tags is applied to the given directory and everything under it — in this case, the root directory. Here, the AllowOverride tag dictates that users aren't allowed to override any options (more on this later). The FollowSymLinks option is enabled, which lets Apache look past symlinks to serve the request, even if the file is outside the directory containing Web files. This means that if a file in your Web directory is a symlink to /etc/passwd, the Web server happily serves the file if asked. With -FollowSymLinks used instead, this feature is disabled, and the same request causes an error to be returned to the client.

This last scenario is a cause for concern on two fronts. The first is a performance matter. If FollowSymLinks is disabled, then Apache must check each component of the filename (directories and the file itself) to make sure they're not symbolic links. This incurs extra overhead in the form of disk activity. A companion option called FollowSymLinksIfOwnerMatch follows the symbolic link if the owner of the file is the same as that of the link. This has the same performance hit as disabling following of symlinks. For best performance, use the options in Listing 2.

Security-conscious readers should be alert by now. Security is always a trade-off between functionality and risk. In this case, the functionality is speed, and the risk is allowing unauthorized access to files on the system. One of the mitigations is that LAMP application servers are generally dedicated to a particular function, and users can't create the potentially dangerous symbolic links. If it's vital to have symbolic link-checking enabled, you can restrict it to a particular area of the file system, as in Listing 3.

Listing 3. Restricting FollowSymLinks to a user's directory

<Directory /> Options FollowSymLinks </Directory> <Directory /home/*/public_html> Options -FollowSymLinks </Directory>

In Listing 3, any public_html directory in a user's home directory has the FollowSymLinks option removed for it and any child directories.

As you've seen, options can be configured on a per-directory basis through the main server configuration. Users can override this server configuration themselves (if permitted by the administrator by the AllowOverrides statement) by dropping a file called .htaccess into a directory. This file contains additional server directives that are loaded and followed on each request to the directory where the .htaccess file resides. Despite the earlier discussion about not having users on the system, many LAMP applications use this functionality to control access and for URL rewriting, so it's wise to understand how it works.

Even though the AllowOverrides statement prevents users from doing anything you don't want them to, Apache must still look for the .htaccess file to see if there is any work to be done. A parent directory can specify directives that are to be processed by requests from child directories, which means Apache must also search each component of the directory tree leading to the requested file. Understandably, this causes a great deal of disk activity on each request.

The easiest solution is to not allow any overrides, which eliminates the need for Apache to check for .htaccess. Any special configurations are then placed directly in httpd.conf. Listing 4 shows the additions to httpd.conf to enable password checking for a user's project directory, rather than putting in a .htaccess file and relying on AllowOverrides.

Listing 4. Moving .htaccess configuration into httpd.conf

<Directory /home/user/public_html/project/> AuthUserFile /home/user/.htpasswd AuthName "uber secret project" AuthType basic Require valid-user </Directory>

If the configuration is moved into httpd.conf and AllowOverrides is disabled, disk usage can be reduced. A user's project may not attract many hits, but consider how powerful this technique is when applied to a busy site.

Sometimes it's not possible to eliminate use of .htaccess files. For example, in Listing 5, where an option is restricted to a certain part of the file system, overrides can also be scoped.

Listing 5. Scoping .htaccess checking

<Directory /> AllowOverrides None </Directory> <Directory /home/*/public_html> AllowOverrides AuthConfig </Directory>

After you implement Listing 5, Apache still looks for .htaccess files in the parent directories, but it stops in the public_html directory because the rest of the file system has the functionality disabled. For example, if a file that maps to /home/user/public_html/project/notes.html is requested, only the public_html and project directories are searched.

One final note about per-directory configurations is in order. Any document about tuning Apache will tell you to disable DNS lookups through the HostnameLookups off directive because trying to reverse-resolve every IP address connecting to your server is a waste of resources. However, any limitations based on hostname force the Web server to perform a reverse lookup on the client's IP address and a forward lookup on the result of that to verify the authenticity of the name. Therefore, it's wise to avoid using access controls based on the client's hostname and to scope them as described when they're necessary.

Persistent connections

When a client connects to a Web server, it's allowed to issue multiple requests over the same TCP connection, which reduces the latency associated with multiple connections. This is useful when a Web page refers to several images: The client can request the page and then all the images over one connection. The downside is that the worker process on the server has to wait for the session to be closed by the client before it can move on to the next request.

Apache lets you configure how persistent connections, called keepalives, are handled. KeepAlive 5 at the global level of httpd.conf allows the server to handle 5 requests on a connection before forcing the connection closed. Setting this number to 0 disables the use of persistent connections. KeepAliveTimeout, also at the global level, determines how long Apache will wait for another request before closing the session.

Handling persistent connections isn't a one-size-fits-all configuration. Some Web sites fare better with keepalives disabled (KeepAlive 0), and some experience a tremendous benefit by having them on. The only solution is to try both and see for yourself. It's advisable, though, to use a low timeout such as 2 seconds with KeepAliveTimeout 2 if you enable keepalives. This ensures that any client wishing to make another request has ample time, and that worker processes aren't idling while waiting for another request that may never come.

Compression

The Web server can compress the output before it's sent back to the client. This results in a smaller page being sent over the Internet at the expense of CPU cycles on the Web server. For those servers that can afford the CPU overhead, this is an excellent way of making pages download faster — it isn't unheard of for pages to be a third of their size after compression.

Images are generally already compressed, so compression should be limited to text output. Apache provides compression through mod_deflate. Although mod_deflate can be simple to turn on, it includes many complexities that the manual is eager to explain. This article doesn't cover the configuration of compression except to provide a link to the appropriate documentation (see the Resources section.)

Tuning PHP

PHP is the engine that runs the application code. You should install only the modules you plan to use and have your Web server configured to use PHP only for script files (usually those ending in .php) and not all static files.

Opcode caching

When a PHP script is requested, PHP reads the script and compiles it into what's called Zend opcode, a binary representation of the code to be executed. This opcode is then executed by the PHP engine and thrown away. An opcode cache saves this compiled opcode and reuses it the next time the page is called. This saves a considerable amount of time. Several opcode caches are available; I've had a great deal of success with eAccelerator.

Installing eAccelerator requires the PHP development libraries on your computer. Because different Linux distributions place files in difference places, it's best to get the installation instructions directly from the eAccelerator Web site (see the Resources section for a link). It's also possible that your distribution has already packaged an opcode cache, and you just have to install it.

Regardless of how you get eAccelerator on your system, there are a few configuration options to look at. The configuration file is usually /etc/php.d/eaccelerator.ini. eaccelerator.shm_size defines the size of the shared memory cache, which is where the compiled scripts are stored. The value is in megabytes. Determining the proper size depends on your application. eAccelerator provides a script to show the status of the cache, which includes the memory usage; 64 megabytes is a good start (eaccelerator.shm_size="64"). You may also have to tweak your kernel's maximum shared memory size if the value you choose isn't accepted. Add kernel.shmmax=67108864 to /etc/sysctl.conf, and run sysctl -p to make the setting take effect. The value for kernel.shmmax is in bytes.

If the shared memory allocation is exceeded, eAccelerator must purge old scripts from memory. By default, this is disabled; eaccelerator.shm_ttl = "60" specifies that when eAccelerator runs out of shared memory, any script that hasn't been accessed in 60 seconds should be purged.

Another popular alternative to eAccelerator is the Alternative PHP Cache (APC). The makers of Zend also have a commercial opcode cache that includes an optimizer to further increase efficiency.

php.ini

You configure PHP in php.ini. Four important settings control how much system resources PHP can consume, as listed in Table 1.

Table 1. Resource related settings in php.ini

Setting	Description	Recommended value
max_execution_time	How many CPU-seconds a script can consume	30
max_input_time	How long (seconds) a script can wait for input data	60
memory_limit	How much memory (bytes) a script can consume before being killed	32M
output_buffering	How much data (bytes) to buffer before sending out to the client	4096

These numbers depend mostly on your application. If you accept large files from users, then max_input_time may have to be increased, either in php.ini or by overriding it in code. Similarly, a CPU- or memory-heavy program may need larger settings. The purpose is to mitigate the effect of a runaway program, so disabling these settings globally isn't recommended. Another note on max_execution_time: This refers to the CPU time of the process, not the absolute time. Thus a program that does lots of I/O and few calculations may run for much longer than max_execution_time. It's also how max_input_time can be greater than max_execution_time

The amount of logging that PHP can do is configurable. In a production environment, disabling all but the most critical logs saves disk writes. If logs are needed to troubleshoot a problem, you can turn up logging as needed. error_reporting = E_COMPILE_ERROR|E_ERROR|E_CORE_ERROR turns on enough logging to spot problems but eliminates a lot of chatter from scripts.

Summary

This article focused on tuning the Web server, both Apache and PHP. With Apache, the general idea is to eliminate extra checks the Web server must do, such as processing the .htaccess file. You must also tune the Multi-Processing Module you're using to balance the system resources used with the availability of idle workers for incoming requests. The best thing you can do for PHP is to install an opcode cache. Keeping your eye on a few resource settings also ensures that scripts don't hog resources and make the system slow for everyone else.

The next and final article in this series will look at tuning the MySQL database. Stay tuned!

Resources

more..

Learn

• "Quantify performance changes using application tracing" (developerWorks, July 2006) shows how to use application tracing to show the effect of configuration changes on Apache.
  
  
• "Using the new memory manager" (developerWorks, March 2007) covers the latest changes to PHP 5.2's handling of memory. PHP is constantly refining its use of system resources.
  
  
• mod_deflate is an Apache module that compresses output on the fly. This can also be done in PHP through output compression.
  
  
• Pre-caching compressed static files such as JavaScript code. CSS is another way to improve performance. Compressing and concatenating all your JavaScript code and CSS is even better.
  
  
• The Apache documentation on Multi-Processing Modules is worth reading to learn about the functionality of each; follow the links to the specific documentation for the MPM you choose.
  
  
• In the developerWorks Linux zone, find more resources for Linux developers.
  
  
• Stay current with developerWorks technical events and Webcasts.

Get products and technologies

• If your distribution doesn't include eAccelerator, the Install From Source instructions will be helpful.
  
  
• The Alternative PHP Cache and Zend Platform are alternatives to eAccelerator.
  
  
• Siege lets you simulate users, so you can find out how much traffic your site can handle.
  
  
• Sooner or later you're going to want to cache certain elements of your site and distribute load across multiple Web servers. Squid in accelerator mode (also known as a reverse proxy) or the Linux Virtual Server Project are excellent tools.
  
  
• Order the SEK for Linux, a two-DVD set containing the latest IBM trial software for Linux from DB2®, Lotus®, Rational®, Tivoli®, and WebSphere®.
  
  
• With IBM trial software, available for download directly from developerWorks, build your next development project on Linux.
  
  

Discuss
Check out developerWorks blogs and get involved in the developerWorks community.

[Source] IBM developerWorks Tuning LAMP systems, Part 1

Tuning LAMP systems, Part 1: Understanding the LAMP architecture

How LAMP systems work, how to measure their performance, and how to tune the underlying operating system

Level: Intermediate

Sean A. Walberg (sean@ertw.com), Senior Network Engineer

31 Mar 2007

Applications using the LAMP (Linux®, Apache, MySQL, PHP/Perl) architecture are constantly being developed and deployed. But often the server administrator has little control over the application itself because it's written by someone else. This series of three articles discusses many of the server configuration items that can make or break an application's performance. This first article covers the LAMP architecture, some measurement techniques, and some basic Linux kernel, disk, and file system tweaks. Successive articles investigate tuning the Apache, MySQL, and PHP components.

Linux, Apache, MySQL, and PHP (or Perl) are the foundation of many Web applications, from to-do lists to blogs to e-commerce sites. WordPress and Pligg are but two common software packages powering high-volume Web sites. This architecture has come to be known simply as LAMP. Almost every distribution of Linux includes Apache, MySQL, PHP, and Perl, so installing the LAMP software is almost as easy as saying it.

This ease of installation gives the impression that the software runs itself, which is simply not true. Eventually the load on the application outgrows the settings that come bundled with the back-end servers and application performance suffers. LAMP installations require constant monitoring, tuning, and evaluation.

Tuning a system has different meanings to different people. This series of articles focuses on tuning the LAMP components -- Linux, Apache, MySQL, and PHP. Tuning the application itself is yet another complex matter. There is a symbiotic relationship between the application and the back-end servers: a poorly tuned server causes even the best application to fail under load, and there's only so much tuning one can do to a server before a badly written application slows to a crawl. Fortunately, proper system tuning and monitoring can point to problems in the application.

The LAMP architecture

The first step in tuning any system is understanding how it works. At the simplest level, a LAMP-based application is written in a scripting language such as PHP that runs as part of the Apache Web server that is running on a Linux host.

The PHP application takes information from the client through the requested URL, any form data, and whatever session information has been captured to determine what it is supposed to do. If needed, the server pulls information from a MySQL database (also running on Linux), combines the information with some Hypertext Markup Language (HTML) templates, and returns it to the client. This process repeats itself as the user navigates the application and also occurs in parallel as multiple people access the system. The flow of data is not one way, however, because the database may be updated with information from the user in the form of session data, statistics collection (including voting), and user-submitted content such as comments or site updates. In addition to the dynamic elements, there are also static elements such as images, JavaScript code, and Cascading Style Sheets (CSS).

After looking at the flow of requests through the LAMP system, you can begin to see the points where slowdowns might occur. The database provides much of the dynamic information, so the client notices any delay in responding to queries. The Web server must be able to execute the scripts quickly and also handle multiple concurrent requests. Finally, the underlying operating system must be in good health to support the applications. Other setups that share files between different servers over the network can also become a possible bottleneck.

Variations on LAMP

LAMP started out as strictly Linux, Apache, MySQL, and PHP or Perl. It is not uncommon, however, to run Apache, MySQL, and PHP on Microsoft® Windows® if Linux isn't your strength. Then again, you can always swap out Apache for something like lighttpd, and you still have a LAMP-style system, albeit one with an unpronounceable acronym. Or you may prefer a different open source database such as PostgreSQL or SQLite, a commercial database such as IBM® DB2®, or even a commercial but free engine like IBM DB2 Express-C.

This article focuses on the traditional LAMP architecture because it's the one I see most often in my travels, and its components are all open source.

Measuring performance

Constant measurement of performance helps in two ways. The first is that measurement helps you spot trends, both good and bad. As a simple example, by watching central processing unit (CPU) usage on a Web server, you can see when it is overloaded. Similarly, watching the total bandwidth used in the past and extrapolating to the future helps you determine when network upgrades are needed. These measurements are best correlated with other measurements and observations. For example, you might determine that when users complain of application slowness, the disks happen to be operating at maximum capacity.

The second use of performance measurements is to determine if tuning has helped the situation or made it worse. You do this by comparing measurements before and after the change is made. For this to be effective, though, only one item should be changed at a time, and the proper metric should be compared to determine the effect of the change. The reason for changing only one thing at a time should be obvious. After all, it is quite possible that two simultaneous changes could counteract each other. The reason for the metrics statement is more subtle.

It is crucial that the metrics you choose to watch reflect on the user of the application. If the goal of a change is to reduce the memory footprint of the database, eliminating various buffers will certainly help, at the expense of query speed and application performance. Instead, one of the metrics should be application response time, which opens up tuning possibilities other than just the database's memory usage.

You can measure application response time in many ways. Perhaps the easiest is with the curl command shown in Listing 1.

Listing 1. Using cURL to measure the response time of a Web site

$ curl -o /dev/null -s -w %{time_connect}:%{time_starttransfer}:%{time_total}\ http://www.canada.com 0.081:0.272:0.779

Listing 1 shows the curl command being used to look up a popular news site. The output, which would normally be the HTML code, is sent to /dev/null with the -o parameter, and -s turns off any status information. The -w parameter tells curl to write out some status information such as the timers described in Table 1:

Table 1. Timers used by curl

Timer	Description
time_connect	The time it takes to establish the TCP connection to the server
time_starttransfer	The time it takes for the Web server to return the first byte of data after the request is issued
time_total	The time it takes to complete the request

Each of these timers is relative to the start of the transaction, even before the Domain Name Service (DNS) lookup. Thus, after the request was issued, it took 0.272 - 0.081 = 0.191 seconds for the Web server to process the request and start sending back data. The client spent 0.779 - 0.272 = 0.507 seconds downloading the data from the server.

By watching curl data and trending it over time, you get a good idea of how responsive the site is to users.

Of course, a Web site is more than just a single page. It has images, JavaScript code, CSS, and cookies to deal with. curl is good at getting the response time for a single element, but sometimes you need to see how fast the whole page loads.

The Tamper Data extension for Firefox (see the Resources section for a link) logs all the requests made by the Web browser and displays the time each took to download. To use the extension, select Tools > Tamper Data to open the Ongoing requests window. Load the page in question, and you'll see the status of each request made by the browser along with the time the element took to load. Figure 1 shows the results of loading the developerWorks home page.

Figure 1. Breakdown of requests used to load the developerWorks home page

Each line describes the loading of one element. Various data are displayed, such as the time the request started, how long it took to load, the size, and the results. The Duration column lists the time the element itself took to load, while the Total Duration column shows how long all the sub elements took. In Figure 1, the main page took 516 milliseconds (ms) to load, but it was 5101 ms before everything was loaded and the entire page could be displayed.

Another helpful mode of the Tamper Data extension is to graph the output of the page load data. Right-click anywhere in the top half of the Ongoing requests window and select Graph all. Figure 2 shows a graphical view of the data from Figure 1.

Figure 2. A graphical view of requests used to load the developerWorks home page

In Figure 2, the duration of each request is displayed in dark blue and is shown relative to the start of the page load. Thus, you can see which requests are slowing down the whole page load.

Despite the focus on page loading times and user experience, it is important not to lose sight of the core system metrics such as disk, memory, CPU, and network. A wealth of utilities are available to capture this information; perhaps the most helpful are sar, vmstat, and iostat. See the Resources section for more information about these tools.

Basic system tweaks

Before you tune the Apache, PHP, and MySQL components of your system, you should take some time to make sure that the underlying Linux components are operating properly. It goes without saying that you've already stripped down your list of running services to only those that you need. In addition to being a good security practice, doing so saves you both memory and CPU cycles.

Some quick kernel tuning

Most Linux distributions ship with buffers and other Transmission Control Protocol (TCP) parameters conservatively defined. You should change these parameters to allocate more memory to enhancing network performance. Kernel parameters are set through the proc interface by reading and writing to values in /proc. Fortunately, the sysctl program manages these in a somewhat easier fashion by reading values from /etc/sysctl.conf and populating /proc as necessary. Listing 2 shows some more aggressive network settings that should be used on Internet servers.

Listing 2. /etc/sysctl.conf showing more aggressive network settings

# Use TCP syncookies when needed net.ipv4.tcp_syncookies = 1 # Enable TCP window scaling net.ipv4.tcp_window_scaling = 1 # Increase TCP max buffer size net.core.rmem_max = 16777216 net.core.wmem_max = 16777216 # Increase Linux autotuning TCP buffer limits net.ipv4.tcp_rmem = 4096 87380 16777216 net.ipv4.tcp_wmem = 4096 65536 16777216 # Increase number of ports available net.ipv4.ip_local_port_range = 1024 65000

Add this file to whatever is already in /etc/sysctl.conf. The first setting enables TCP SYN cookies. When a new TCP connection comes in from a client by means of a packet with the SYN bit set, the server creates an entry for the half-open connection and responds with a SYN-ACK packet. In normal operation, the remote client responds with an ACK packet that moves the half-open connection to fully open. An attack called the SYN flood ensures that the ACK packet never returns so that the server runs out of room to process incoming connections. The SYN cookie feature recognizes this condition and starts using an elegant method that preserves space in the queue (see the Resources section for full details). Most systems have this enabled by default, but it's worth making sure this one is configured.

Enabling TCP window scaling allows clients to download data at a higher rate. TCP allows for multiple packets to be sent without an acknowledgment from the remote side, up to 64 kilobytes (KB) by default, which can be filled when talking to higher latency peers. Window scaling enables some extra bits to be used in the header to increase this window size.

The next four configuration items increase the TCP send and receive buffers. This allows the application to get rid of its data faster so it can serve another request, and it also improves the remote client's ability to send data when the server gets busier.

The final configuration item increases the number of local ports available for use, which increases the maximum number of connections that can be served at a time.

These settings become effective at next boot or the next time sysctl -p /etc/sysctl.conf is run.

Configure disks for maximum performance

Disks play a vital role in the LAMP architecture. Static files, templates, and code are served from disk, as are the data tables and indexes that make up the database. Much of the tuning to follow, especially that pertaining to the database, focuses on avoiding disk access because of the relatively high latency it incurs. Therefore, it makes sense to spend some time optimizing the disk hardware.

The first order of business is to ensure that atime logging is disabled on file systems. The atime is the last access time of a file, and each time a file is accessed, the underlying file system must record this timestamp. Because atime is rarely used by systems administrators, disabling it frees up some disk time. This is accomplished by adding the noatime option in the fourth column of /etc/fstab. Listing 3 shows an example configuration.

Listing 3. A sample fstab showing how to enable noatime

/dev/VolGroup00/LogVol00 / ext3 defaults,noatime 1 1 LABEL=/boot /boot ext3 defaults,noatime 1 2 devpts /dev/pts devpts gid=5,mode=620 0 0 tmpfs /dev/shm tmpfs defaults 0 0 proc /proc proc defaults 0 0 sysfs /sys sysfs defaults 0 0 LABEL=SWAP-hdb2 swap swap defaults 0 0 LABEL=SWAP-hda3 swap swap defaults 0 0

Only the ext3 file systems have been modified in Listing 3 because noatime is helpful only for file systems that reside on a disk. A reboot is not necessary to effect this change; you only need to remount each file system. For example, to remount the root file system, run mount / -o remount.

A variety of disk hardware combinations are possible, and Linux doesn't always reliably detect the optimal way to access the disks. The hdparm command is used to get and set the methods used to access IDE disks. hdparm -t /path/to/device performs a speed test that you can use as a benchmark. For the most reliable results, the system should be idle when you run this command. Listing 4 shows a speed test being performed on hda.

Listing 4. A speed test being performed on /dev/hda

# hdparm -t /dev/hda /dev/hda: Timing buffered disk reads: 182 MB in 3.02 seconds = 60.31 MB/sec

As the test shows, the disks are reading data at around 60 megabytes (MB) per second.

Before delving into some of the disk tuning options, a warning is in order. The wrong setting can corrupt the file system. Sometimes you get a warning that the option isn't compatible with your hardware; sometimes you don't. For this reason, test settings thoroughly before putting a system into production. Having standard hardware across all your servers helps here too.

Table 2 lists some of the more common options.

Table 2. Common options for hdparm

Option	Description
-vi	Query the drive to determine which settings it supports and which settings it is using.
-c	Query/enable (E)IDE 32-bit I/O support. hdparm -c 1 /dev/hda enables this.
-m	Query/set multiple sectors per interrupt mode. If the setting is greater than zero, up to that number of sectors can be transferred per interrupt.
-d 1 -X	Enable direct memory access (DMA) transfers and set the IDE transfer mode. The hdparm man page details the numbers that may go after the -X. You should need to do this only if -vi shows you're not using the fastest mode.

Unfortunately for Fiber Channel and Small Computer Systems Interface (SCSI) systems, tuning is dependent on the particular driver.

You must add whichever settings you find useful to your startup scripts, such as rc.local.

Network file system tuning

The network file system (NFS) is a way to share disk volumes across the network. NFS is helpful to ensure that every host has a copy of the same data and that changes are reflected across all nodes. By default, though, NFS is not configured for high-volume use.

Each client should mount the remote file system with rsize=32768,wsize=32768,intr,noatime to ensure the following:

• Large read/write block sizes are used (up to the specified figure, in this case 32KB).
• NFS operations can be interrupted in case of a hang.
• The atime won't be constantly updated.

You can put these settings in /etc/fstab, as shown in Listing 3. If you use the automounter, these go in the appropriate /etc/auto.* file.

On the server side, it is important to make sure there are enough NFS kernel threads available to handle all your clients. By default, only one thread is started, though Red Hat and Fedora systems start at 8. For a busy NFS server, you should push this number higher, such as 32 or 64, to start. You can evaluate your clients to see if there was blockage with the nfsstat -rc command, which shows client Remote Procedure Call (RPC) statistics. Listing 5 shows the client statistics for a Web server.

Listing 5. Showing a NFS client's RPC statistics

# nfsstat -rc Client rpc stats: calls retrans authrefrsh 1465903813 0 0

The second column, retrans, is zero, showing that no retransmissions were necessary since the last reboot. If this number is climbing, then you should consider adding more NFS kernel threads. This is done by passing the number of threads desired to rpc.nfsd, such as rpc.nfsd 128 to start 128 threads. You can do this at any time. Threads are started or destroyed as necessary. Again, this should go in your startup scripts, preferably in the script that starts NFS on your system.

A final note on NFS: Avoid NFSv2 if you can because performance is much less than in v3 and v4. This is not an issue in modern Linux distributions, but check the output of nfsstat on the server to see if any NFSv2 calls are being made.

Resources

more..

Learn

• "Easy system monitoring with SAR" (developerWorks, February 2006) is a guide to keeping track of key system metrics using sar.
  
  
• "Expose Web performance problems with the RRDtool" (developerWorks, March 2006) is a tutorial by Sean that expands on the cURL technique and graphs the data for long-term analysis.
  
  
• In Monitoring Virtual Memory with vmstat (Linux Journal, October 2005), you learn how to observe paging activity on your Linux system.
  
  
• If NFS is new to you, To Protect and Serve: Providing Data to Your Cluster (Prentice Hall Professional Technical Reference, February 2005) is a good introduction to both NFS and the automounter, which helps in large-scale NFS deployments.
  
  
• "TCP and Linux' Pluggable Congestion Control Algorithms" (Linux Gazette, February 2007) discusses how to try out the many algorithms to implement TCP congestion control that Linux supports, and, more importantly, the importance and impact of loss and delay on your network sessions.
  
  
• TCP SYN cookies were mentioned earlier as a defense against denial of service attacks involving SYN floods. Wikipedia describes their implementation. It's a brilliant idea.
  
  
• In the developerWorks Linux zone, find more resources for Linux developers.
  
  
• Stay current with developerWorks technical events and Webcasts.

Get products and technologies

• The Tamper Data extension for Firefox allows you to view and modify HTTP headers on the fly and to graph the loading of page elements.
  
  
• Order the SEK for Linux, a two-DVD set containing the latest IBM trial software for Linux from DB2®, Lotus®, Rational®, Tivoli®, and WebSphere®.
  
  
• With IBM trial software, available for download directly from developerWorks, build your next development project on Linux.
  
  

Discuss
Check out developerWorks blogs and get involved in the developerWorks community.

[Howto] Useful commands for the Linux command line

[Source] HowToForge - Useful Linux Commands

arch

Outputs the processor architecture.

$ arch

i686

cat

Outputs the contents of a file.

$ cat lorem.txt

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

cd

Change the working directory.

$ cd /bin

chgrp

Change group ownership of files.

$ chgrp games moo.txt

chmod

Change access permissions of files.

$ chmod +x helloworld

chown

Change file owner and group.

# chown root lorem.txt

cksum

Print CRC checksum and byte counts of each file.

$ cksum lorem.txt moo.txt

3570240675 453 lorem.txt
4294967295 0 moo.txt

cp

Copies a file.

$ cp lorem.txt copy_of_lorem.txt

date

Outputs the current date and time.

$ date

Sat Mar  3 12:07:09 GMT 2007

df

Reports the amount of disk space used and available on filesystems.

$ df

Filesystem           1K-blocks      Used Available Use% Mounted on<br>
/dev/simfs            39845888    218048  39627840   1% /

dir

List directory contents.

$ dir

copy_of_lorem.txt  lorem.txt  moo.txt  www

du

Estimate file space usage.

$ du -h /bin

7.8M    /bin

echo

Display a line of text.

$ echo foobar

foobar

exit

Cause the shell to exit.

$ exit

fgrep

Print lines matching a pattern in a file.

$ fgrep "irure dolor" lorem.txt

commodo consequat. Duis aute irure dolor in reprehenderit in voluptate

find

Search for files in a directory hierarchy.

$ find hello*

hello_world
hello_world.c

free

Display amount of free and used memory in the system.

$ free

             total       used       free     shared    buffers     cached
Mem:       8299892    8287708      12184          0    2641772    1731236
Low:       3581300    3572764       8536
High:      4718592    4714944       3648
-/+ buffers/cache:    3914700    4385192
Swap:      8193140    2335664    5857476

grep

Print lines matching a pattern.

$ grep -i apple fruitlist.txt

apple

groups

Outputs the user groups of which your account belongs to.

$ groups

games users

head

Output the first part of files.

$ head -2 lorem.txt

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim

hostname

Outputs the machines hostname on the network.

$ hostname

anapnea.net

id

Outputs user id, group id, and groups of your account.

$ id

uid=1478(smallfoot) gid=100(users) groups=35(games),100(users)

kill

End a process.

$ kill -9 18298

-bash: kill: (18298) - Operation not permitted

killall

Kill processes by name.

$ killall irssi

irssi(18298): Operation not permitted
irssi(13372): Operation not permitted
irssi(22048): Operation not permitted
irssi: no process killed

last

Show listing of last logged in users.

$ last -n 3

alice    pts/6        192.0.34.166     Fri May 18 16:17   still logged in
bob      pts/2        64.233.183.103   Fri May 18 16:17   still logged in
clare    pts/6        72.5.124.61      Fri May 18 15:54 - 15:55  (00:01)

ldd

Print shared library dependencies.

$ ldd /bin/bash

        libncurses.so.5 => /lib/libncurses.so.5 (0x40023000)
       libdl.so.2 => /lib/libdl.so.2 (0x40065000)
       libc.so.6 => /lib/libc.so.6 (0x40069000)
       /lib/ld-linux.so.2 (0x40000000)

ln

Make links between files.

$ ln -s data.txt symlink.txt

logname

Print user's login name.

$ logname

smallfoot

ls

List directory contents.

$ ls

copy_of_lorem.txt  lorem.txt  moo.txt  www

man

Opens the manual page for a software or function.

$ man bash

md5sum

Outputs the MD5 hash sum of a file.

$ md5sum lorem.txt

56da9e37259af34345895883e6fd1a27  lorem.txt

mkdir

Makes a directory.

$ mkdir foobar

mv

Moves a file.

$ mv lorem.txt ipsum.txt

nl

Number lines of files.

$ nl lorem.txt

     1  Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
    2  tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim
    3  veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea
    4  commodo consequat. Duis aute irure dolor in reprehenderit in voluptate
    5  velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint
    6  occaecat cupidatat non proident, sunt in culpa qui officia deserunt
    7  mollit anim id est laborum.

nm

List symbols from object files.

$ nm hello_world

080494a0 D _DYNAMIC
0804956c D _GLOBAL_OFFSET_TABLE_
08048474 R _IO_stdin_used
        w _Jv_RegisterClasses
08049490 d __CTOR_END__
0804948c d __CTOR_LIST__
08049498 d __DTOR_END__
...

od

Dump files in octal and other formats.

$ od -t x /bin/sh

2376640 00098020 000054d4 00000000 00000000
2376660 00000020 00000000 000000c7 00000008
2376700 00000003 080e6500 0009d4f4 00004ae8
...

pidof

Find the process ID of a running program.

$ pidof fetchmail

22392

ping

Pings a host.

$ ping -c 2 127.0.0.1

PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data.
64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.048 ms
64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.052 ms

--- 127.0.0.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.048/0.050/0.052/0.002 ms

ps

Outputs running processes.

$ ps

  PID TTY          TIME CMD
21542 pts/12   00:00:00 bash
27706 pts/12   00:00:00 ps

pstree

Display a tree of processes.

$ pstree

init-+-2*[BitchX]
    |-3*[bash---sleep]
    |-fetchmail
    |-screen-+-bash---irssi
    |        `-bash---ctorrent
    |-screen-+-bash---lisp.run
    |        |-bash---vi
    |        |-2*[bash]
    |        `-bash---lynx
    |-2*[screen---bash---irssi]
    |-screen---irssi
    |-screen---bash
    |-screen-+-bash
    |        `-irssi
    |-skjerm---irssi
    |-sshd-+-5*[sshd---sshd---bash---irssi]
    |      |-8*[sshd---sshd---bash]
    |      |-sshd---sshd---bash---screen
    |      |-sshd---sshd
    |      `-sshd---sshd---bash---pstree
    `-syslog-ng

pwd

Outputs the name of current working directory.

$ pwd

/home/smallfoot

rm

Removes a file or directory.

$ rm lorem.txt

rmdir

Removes a directory.

$ rmdir foobar

sed

Stream editor for filtering and transforming text.

$ echo "My cat's name is Bob" | sed -e 's/Bob/Mittens/g'

My cat's name is Mittens

sha1sum

Outputs the SHA1 hash sum of a file.

$ sha1sum lorem.txt

c942ddebd142ec8bacac9213d48096e74bab4957  lorem.txt

shutdown

Bring the system down in a secure way. All logged-in users are notified that the system is going down.

$ shutdown now

size

List section sizes and total size.

$ size /bin/bash

   text    data     bss     dec     hex filename
621233   22712   19176  663121   a1e51 /bin/bash

stat

Outputs file status.

$ stat lorem.txt

  File: `lorem.txt'
 Size: 453             Blocks: 8          IO Block: 4096   regular file
Device: 77h/119d        Inode: 27312217    Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1478/smallfoot)   Gid: (  100/   users)
Access: 2007-03-03 12:24:39.000000000 +0000
Modify: 2007-03-03 12:24:39.000000000 +0000
Change: 2007-03-03 12:24:39.000000000 +0000

strings

Print the strings of printable characters in files.

$ strings hello_world

/lib/ld-linux.so.2
_Jv_RegisterClasses
__gmon_start__
libc.so.6
puts
_IO_stdin_used
__libc_start_main
GLIBC_2.0
PTRh%
[^_]
Hello World!

tail

Output the last part of files.

$ tail -2 lorem.txt

occaecat cupidatat non proident, sunt in culpa qui officia deserunt
mollit anim id est laborum.

talk

Talk to another user.

$ talk bob Lookout for the dopefish!

touch

Change a file's access and modification timestamps. If file does not exist, create it.

$ touch lorem.txt

tty

Outputs the name of the current terminal.

$ tty

/dev/pts/16

uname

Outputs operating system, hostname, kernel version, date and timp, and processor.

$ uname -a

Linux anapnea.net 2.6.9 #1 SMP Wed Jul 19 16:24:18 MSD 2006 i686 Intel(R) Xeon(TM) CPU 2.80GHz GenuineIntel GNU/Linux

uptime

Outputs the system uptime.

$ uptime

 14:50:26 up 7 days, 17:52, 18 users,  load average: 0.08, 0.02, 0.01

users

Print the user names of users currently logged in to the current host.

$ users

alice bob charlie eve

vdir

List directory contents.

$ vdir

total 8
-rw-r--r-- 1 smallfoot users 453 Mar  3 12:32 copy_of_lorem.txt
-rw-r--r-- 1 smallfoot users 453 Mar  3 12:24 lorem.txt
-rw-r--r-- 1 smallfoot users   0 Mar  3 12:32 moo.txt
lrwxr-xr-x 1 root      root   18 Feb 27 19:33 www -> /var/www/smallfoot

w

Show who is logged on and what they are doing.

$ w

 12:14:30 up 5 days, 15:16, 19 users,  load average: 0.00, 0.00, 0.00
USER     TTY        LOGIN@   IDLE   JCPU   PCPU WHAT
charlie  pts/0     Fri21    3:26m  2.52s  2.52s irssi
alice    pts/2     Wed17   30:21m  0.00s  0.00s -bash
emma     pts/4     11:37   36:57   0.00s  0.00s -bash
frank    pts/5     11:48   11:03   0.00s  0.00s -bash
smallfoo pts/12    12:01    0.00s  0.04s  0.01s w

wall

Send a message to everybody's terminal.

$ wall next week we change the server for a new one

wc

Counts lines in a file.

$ wc -l lorem.txt

7 lorem.txt

whatis

Search the whatis database for complete words.

$ whatis bash

bash                 (1)  - GNU Bourne-Again SHell
bash [builtins]      (1)  - bash built-in commands, see bash(1)

who

Outputs who is currently logged into the system.

$ who

charlie  pts/0        Mar  2 21:37 (xtreme-11-65.acme.com)
alice    pts/2        Feb 28 17:48 (147.21.16.3)
emma     pts/4        Mar  3 11:37 (32.84-48-181.uac.com)
frank    pts/5        Mar  3 11:48 (port-212-202-233-2.foobar.org)
smallfoot pts/12       Mar  3 12:01 (c-12776f4.cust.example.net)

whereis

Locate the binary, source, and manual page files for a command.

$ whereis bash

bash: /bin/bash /etc/bash /usr/share/man/man1/bash.1.gz

whoami

Outputs your username / the name of your account.

$ whoami

smallfoot