Skip to content

Notes on move to Nginx

January 16, 2013

We are trying out with Nginx, instead of Apache. Feel a bit sad, sort of like, we betrayed Apache httpd. But the reason, was Memory usage. Also the number of processes which Apache spawned. On a single EC2 instance we have Nginx serving the same number of requests with just 4 child processes compared to Apache’s numerous (some times over 50).

Also because Apache embeds the PHP engine, and hence per child process its memory goes up to 25 MB (from 8 MB without the PHP module). But Nginx works with php-fpm which is a much better solution, in that there is a separate php-fpm server for processing PHP stuff. And hence Nginx processes remain lightweight in terms of memory as well.

Below are list of references, which can be very helpful in the installation, and also give a very good idea of Nginx philosophy:

1. Very useful article on Nginx:
http://arstechnica.com/gadgets/2012/11/how-to-set-up-a-safe-and-secure-web-server/

2. Techincal article on Nginx:
http://www.aosabook.org/en/nginx.html

3. Martin Fjordvald has written some really great articles on Nginx
a. A primer: http://blog.martinfjordvald.com/2010/07/nginx-primer/
b. Nginx configuration model: http://blog.martinfjordvald.com/2012/08/understanding-the-nginx-configuration-inheritance-model/
c. Optimixing Nginx for high traffic: http://blog.martinfjordvald.com/2011/04/optimizing-nginx-for-high-traffic-loads/

4. Pages from Nginx site (http://wiki.nginx.org/):
a. http://nginx.org/en/docs/install.html

5. Nginx sticky module:
http://code.google.com/p/nginx-sticky-module/

6. Helped in setting up nginx with Jetty config. For passing on Client IPs upto Jetty:
http://grokbase.com/t/gg/liftweb/126vptpdwh/lift-how-to-get-request-real-ip-in-liftweb-nginx-jetty

7. Explains nicely how simply changing httpd Listen to 85 (anything other than 80) does not work, with wordpress. I am quite angry with them for this stupid config need. They should fix it.
http://jumptuck.com/2009/01/09/wordpress-on-non-standard-port-not-port-80/

8. php with nginx
– Nice and simple one to get started: http://www.howtoforge.com/installing-nginx-with-php5-and-php-fpm-and-mysql-support-lemp-on-ubuntu-12.04-lts
– Gives a good background: http://blog.digitalstruct.com/2010/07/12/getting-started-with-nginx-and-php-fpm/

9. Explains nginx working with wordpress
http://wiki.nginx.org/WordPress

You may skip the specifics below:

Below are some important tech notes (for remembering sake):

A1. Steps to Install Using Compile:

1. Get Nginx source from http://nginx.org/download/nginx-1.3.10.tar.gz

2. Nginx’s http rewrite module needs the PCRE library, so we get it using:
wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.32.tar.gz

3. Get zlib library. Nginx uses it gor gzipping content:
wget http://zlib.net/zlib-1.2.7.tar.gz

4. Get nginx-sticky-module. Used for stickiness to upstream Jettys
wget http://nginx-sticky-module.googlecode.com/files/nginx-sticky-module-1.1.tar.gz

5. Get the OpenSSL sources. Used for HTTPS. Also needed by the nginx-sticky-module for hashing.
wget http://www.openssl.org/source/openssl-1.0.1c.tar.gz

6. Unzip all sources at parallel locations in the same directory

7. Configure Nginx
NOTE: This build does not support SSL. For that we need to also get openssl sources, and build with appropriate options (–with-http_ssl_module . Instructions in install page[4])

./configure –with-pcre=../pcre-8.32 –with-zlib=../zlib-1.2.7 –with-http_ssl_module –with-openssl=../openssl-1.0.1c –add-module=../nginx-sticky-module-1.1

8. Build and install:
make && make install

9. Making nginx a service. When we compile and not use the linux update commands yum update etc.
This link is helpful: http://www.ruby-forum.com/topic/169587

A2: Install using ‘yum install’

1. yum install nginx
(installed a nginx 1.2.5 version, with a script in /etc/init.d/)
NOTE: We discard this option, as we want the sticky module, which needs to be compiled with nginx source)

 

B. Configuration of Nginx

1. Look at the modified nginx.conf

NOTE: nginx -s reload can be used to reload the new config without a restart

2. Add nginx to root path

3. The forwarding to jetty should be done like following:
proxy_pass http://jetty/travel;

proxy_set_header  X-Real-IP  $remote_addr;
proxy_set_header  Host $http_host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

NOTE: This along with ‘forwarded’ flag in jetty (see point below), which makes is use the x-Forwarded-For header, is used to get the effect

4. The jettys should have the following in config (the file is myjetty.xml)
<Configure id=”Server”>

<Call name=”addConnector”>
<Arg>
<New>
<Set name=”port”>8080</Set>
<Set name=”forwarded”>true</Set>
</New>
</Arg>
</Call>

</Configure>

5. Log rotation
a. Created scripts in /home/ninetydi/tools/bin/rotate_nginx_log.sh (as per http://wiki.nginx.org/LogRotation)
b. Added crontab entries (for root) which runs daily around 5 PM (kept them 5 minutes apart, as zipping of old logs may take time)

6. gzip – by default it only gzips html. Also does not zipped proxied content.
But using directives like ‘gzip_proxied any’ helps. See the nginx.conf file for directives below ‘gzip on’

7. The mobile directives are added in a separate file mobile_rewrite.conf and included in nginx.conf

8. Running PHP stuff. We use php-fpm (PHP FastCGI Process Manager)
a. yum install php-fpm
b. chkconfig php-fpm on
d. service pgp-fm start
e. update nginx.conf to have a ‘location ~\.php {…}’ section (see conf file
f. file fastcgi_params (found in the nginx conf dir) should have this extra line
fastcgi_param  SCRIPT_FILENAME    $document_root$fastcgi_script_name;
g. Restart nginx
h. Test: put a small php_info.php file to test it out (http://<domain>/php_info.php)

9. Very important. Else gives 403 (forbidden) error.
Add the following line above the location part (in http or server context)

index  index.html index.htm index.php index.jsp;

10. we customize files 50x.html and 404.html, in nginx html directory, to have our site specific error messages

11. Difference in nginx.conf on uiec2 and slv1 .
– blog section on uiec2’s config is the main one. slv1, just forwards the blog to uiec2
– travelagent, twitter etc. fall to local jettys on uiec2. slv1, just forwards to uiec2

 

C. Running and making it a service

1. Move nginx config and logging to EBS
a) mv /usr/local/nginx/conf/* /vol/etc/nginx/conf
b) Add the following entries to /etc/fstab.
/vol/etc/nginx/conf /usr/local/nginx/conf     none bind
/vol/log/nginx /usr/local/nginx/logs     none bind

2. Stop apache
a) Change all the httpd.conf ports from 80 to 85
b) Everything should work except the wordpress because it needs port specification (see elsewhere in this doc)!
c) Stop httpd

3. Edit /etc/init.d/mountec2vol
a. (backup) cp mountec2vol mountec2vol.httpd
b. edit mountec2vol to mount nginx conf and start nginx instead of httpd on uiec2 and slvs (for the present we leave wsec2 with httpd)

IMPORTANT NOTE: After config change, if we use nginx -s reload, the child processes start dumping core continuously (restart after signal 11). So we need to restart the master.

Advertisements

From → Uncategorized

Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: