Archive for the ‘Site’ Category

Easy delivery with mstmp and GMail

Friday, January 23rd, 2009

At the moment, I really don’t feel like setting up a full-fledged MTA such as sendmail, postfix or qmail. I want to take the simple course, basically because I’m lazy. Fortunately, there are a variety of simple SMTP ‘relays’ out there such as ssmtp and esmtp. Some network officianatos may consider this to be re-inventing the wheel, but then again, I’m sure glad that my car doesn’t roll on stone cylinders.

After some consideration, I chose to go with msmtp. I like its flexible configuration, and it’s just the right size for the job (with room to grow). The major thing I was looking for was STARTTLS support. I wasn’t so concerned with the trust files and certificates, I just had a need to support GMail’s minimum requirements. Yes, msmtp gives you that and the whole 9 yards, for when I need them all.

msmtp Configuration for GMail

With a combination of their official configuration example plus a few targeted suggestions from Grey Bearded Geek’s take at ssmtp, I came up with the following:

# Set default values for all following accounts.
defaults

host  smtp.gmail.com
port  587
timeout  off
protocol  smtp
domain  localhost

# /usr/bin/msmtp --version
#  plain cram-md5 digest-md5 gssapi external login ntlm
auth  on
user  GMAIL-USER
password  GMAIL-PASSWD

tls  on
tls_starttls  on
# WARNING: When the checks are disabled, TLS/SSL sessions will be vulnerable to man-in-the-middle attacks
tls_certcheck  off

logfile  /var/log/msmtp.log
###syslog  on

# A system wide configuration is optional.
# If it exists, it usually defines a default account.
# This allows msmtp to be used like /usr/sbin/sendmail.
account default

# Construct envelope-from addresses of the form "user@oursite.example".
###auto_from on
from  USER@DOMAIN
maildomain  DOMAIN

You plop in the GMAIL-USER and GMAIL-PASSWD, and you’re good to go.

Custom From: Address

I soon learned that the from and maildomain settings are irrelevant; Google will not arbitrarily change the From: header of your mail. That makes sense. So the mail will appear as if it’s coming from you, personally. Well, it turns out that there’s a few things you can do to get around that.

  • Create yourself a dedicated GMail account. Now you have isolated your soon-to-be-wildly-popular start-up’s e-mail account from your personal one.
  • Follow the instructions on adding a custom From: address to your account. I had to use the older version of the GMail interface to do so. Google will verify that you own the address — you’d better be able to receive mail at that address — and then you can make it your default.

    GMail will now send your mail as it were coming from that address, but it will do so without providing an alias.

  • When sending your outbound mail, you can include the following headers:
    From:  ALIAS
    Reply-To:  ALIAS <USER@DOMAIN>

    Google will respect the ALIAS portion of the From: address, though not the address itself. The Reply-To: is optional, but respected in its entirety (alias and address).

Works like a charm.

Wordpress and PmWiki under nginx

Friday, January 23rd, 2009

I recently double-checked my nginx configuration against the one that Elastic Dog has so proudly featured. I’m very glad that I did — they provided me with a better understanding of the if / test capabilities of the syntax.

That being said, it still needed some adjustments …

WordPress

I’m currently running WordPress 2.7 under nginx 0.7.27. Here’s my end configuration:

server {
	include  extra/proxy.conf;
	include  mime.types;

	listen  80;
	server_name  blog.DOMAIN.NAME;

	access_log  logs/LOGFILE-access.log;
	###error_log  logs/LOGFILE-error.log  debug;
	error_log  logs/LOGFILE-error.log;

	# Settings | General | WordPress address (URL)
	location /WP-CONTEXT {
		# physical location on server
		root  /PATH/TO/WP-DIR;
		index  index.php index.html index.htm;

		# remove virtual path, make it root-relative
		rewrite  ^/WP-CONTEXT(.*)$  $1;

		# this serves static files that exist without running other rewrite tests
		if (-f $request_filename) {
			expires 30d;
			break;
		}

		# addresses '/wp-admin/' and similar vein
		if (-f $request_filename/index.php) {
			# produces '...//index.php', acceptable loss
			rewrite  ^(.+)$  /WP-CONTEXT$1/index.php  last;
		}

		# this sends all non-existing file or directory requests to index.php
		if (!-e $request_filename) {
			rewrite  ^(.+)$  /WP-CONTEXT/index.php?q=$1  last;
		}
	}

	location ~ \.php$ {
		# remove virtual path, make it root-relative
		rewrite  ^/WP-CONTEXT(.*)$  $1;

		fastcgi_pass  fastcgi_cluster;
		fastcgi_index index.php;
		# physical location on server
		fastcgi_param SCRIPT_FILENAME /PATH/TO/WP-DIR$fastcgi_script_name;
		include  fastcgi_params;
	}

	# Settings | General | Blog address (URL)
	location / {
		# WordPress handles this perfectly as-is
		fastcgi_pass  fastcgi_cluster;
		# physical location on server
		fastcgi_param SCRIPT_FILENAME /PATH/TO/WP-DIR/index.php;
		include  fastcgi_params;
	}
}

I’ll provide the content of extra/proxy.conf and fastcgi_params down below — they won’t surprise you — plus the configuration for my upstream fastcgi_cluster.

The purpose of DOMAIN.NAME and LOGFILE is obvious, so let’s skip to the useful stuff.

/PATH/TO/WP-DIR

This is simply the fully-qualified path to the directory where you have installed WordPress. Big shock, I know. I put mine in ‘/var/www/wordpress’.

WP-CONTEXT

Specifically, I’m referencing the ‘WordPress address (URL)’ capture block.

Wordpress 2.7 supports a differentiation between the root context of your blog and the root context of the WordPress resources themselves. I’ve taken this approach … the URL of this blog post is root-relative to my virtual hostname, but if you do a View Source you’ll see:

<link rel="stylesheet" href="http://blog.cantremember.localhost/wordpress/wp-content/themes/cantremember/style.css" type="text/css" media="screen" />
...
<link rel="pingback" href="http://blog.cantremember.localhost/wordpress/xmlrpc.php" />

It’s a nice-to-have, and in many ways allows the configuration to be somewhat easier. In WordPress Admin, under Settings | General, I have configured:

  • WordPress address (URL) = http://blog.cantremember.localhost/wordpress
  • Blog address (URL) = http://blog.cantremember.localhost

So my WP-CONTEXT is ‘wordpress’.

Usage

Here are the core differentiations between my config and the Elastic Dog one:

My core two sections are the ones with WP-CONTEXT. Before doing anything, I make the $request_filename context-less, so that it’s corrected relative to root. Granted, I could have skipped that step because I used ‘wordpress’ for each, but that doesn’t make for as good an example, and regex’s aren’t that expensive (don’t they have dedicated chips for them by now?).

I was having issues when WordPress wanted to take me to the Admin screen. It used the shortcut ‘/WP-CONTEXT/wp-admin‘, which is great if you’re not doing all this fancy re-writing and fastcgi_index can take over. But we are being fancy. That’s why the $request_filename/index.php text exists. It works like a charm, although there may be a more efficient way to do this.

And here is where it became an advantage to differentiate between blog URLs and WordPress resources. I’ve chosen to make my permalinks dateless — /%postname%/ . Call me crazy, but I like the way it looks on Laughing Squid. Given that’s the case, it’s hard to differentiate between ‘/some-permalink/’ and ‘/wp-admin/’. Splitting them off with the ‘wordpress’ context made this possible.

The final context-less ‘Blog address (URL)’ capture block is exactly what you’d expect.

PmWiki

I liked the simplicity and capabilities of PmWiki 2.2.0. It’s an easier decision, since I have no intention of being a grand-scale collective document facility. PmWiki is a powerful and flexible implementation with a lot of great processing directives that you can embed in a page. Yet that also makes security something of a concern (as some reviewers will point out as well). Global multi-tier password auth is available, and user-based auth is available as necessary.

This configuration is a natural extension of the WordPress one above:

server {
	include  extra/proxy.conf;
	include  mime.types;

	listen  80;
	server_name  wiki.DOMAIN.NAME;

	access_log  logs/LOGFILE-access.log;
	###error_log  logs/LOGFILE-error.log  debug;
	error_log  logs/LOGFILE-error.log;

	# PHP execution only
	location ~ \.php$ {
		fastcgi_pass  fastcgi_cluster;
		fastcgi_index pmwiki.php;
		# physical location on server
		fastcgi_param SCRIPT_FILENAME /PATH/TO/PMWIKI-DIR$fastcgi_script_name;
		include  fastcgi_params;
	}

	location / {
		# physical location on server
		root  /PATH/TO/PMWIKI-DIR;
		index  index.php index.html index.htm;

		# this serves static files that exist without running other rewrite tests
		if (-f $request_filename) {
			expires 30d;
			break;
		}

		# this sends all non-existing file or directory requests to index.php
		if (!-e $request_filename) {
			# ? => &, then remove the leading /
			rewrite  ^(.*)\?(.*)$  $1&$2;
			rewrite  ^/(.+)$  /?n=$1  last;
		}
	}
}

Everything here is obvious, including /PATH/TO/PMWIKI-DIR. Mine is ‘/var/www/pmwiki’. Here’s the lowdown:

In the *.php capture block, you’ll see that the default script is pmwiki.php. I had created a symlink to rename it index.php, but after my config re-adjustment, that became obsolete.

The non-existing file test will be triggered by the following requests:

  • /Main/HomePage
  • /Main/HomePage?action=edit

Those URLs exist because I’m leveraging a feature called $EnablePathInfo. The referenced documentation doesn’t do it justice … this feature allows me to have bare Group/Name URLs, much like I’m doing with my bare blog URLs. I’ll just say that I’m being SEO-minded and leave it at that.

Turning on that feature informs PmWiki to generate the URLs in that format, and it also makes the PHP script capable of parsing the CGI headers to do-the-right-thing. My original configuration required me to perform the following override hack:

include  fastcgi_params;
fastcgi_param SCRIPT_NAME '';

But the revised configuration above simply re-writes the URL into the standard '?n=‘ format and the script never has to deal with CGI headers. The only other rewrite considerations were to transform any querystring ‘?’ into ‘&’ and to remove the leading ‘/’ from the Group/Name combo.

Hooray!

Supporting Configuration

For all means and purposes, I’m using nginx’s default fastcgi_params.

This is extra/proxy.conf, derived from their NginxFullExample, with notes-to-self intact:

# can't set this here!
###proxy_redirect          default;

proxy_set_header        Host            $host;
proxy_set_header        X-Real-IP       $remote_addr;
proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;

#client_max_body_size    10m;
#client_body_buffer_size 128k;

proxy_connect_timeout   15;
proxy_send_timeout      90;
proxy_read_timeout      30;

# turn off for Comet!
proxy_buffering         on;
proxy_buffers           32 8k;

proxy_ignore_client_abort  on;

# for caching, via Last-Modified:
#proxy_store          on;
#proxy_store_access   user:rw  group:rw  all:r;
#proxy_temp_path      /data/temp;

This is fastcgi_cluster, which is just a simple example of how to do clustering:

upstream fastcgi_cluster {
	# sticky by IP
	###ip_hash;

	# max_fails=3  fail_timeout=15s  weight=2
	# down  backup
	server  127.0.0.1:PORT-1;
	server  127.0.0.1:PORT-2;
	server  127.0.0.1:PORT-3;
}

I tried using some of the additional server setting features — commented out above — but they weren’t working in my build of 0.7.27. I can live without them at the moment, but the upstream block capabilities are quite powerful.

I’m running 3 FastCGI instances, each with 5 worker threads. Again, good enough for government work. I custom-built fcgi on OS X, but for my AWS Fedora Core 8 image I just went with spawn-fcgi that comes along with the lighttpd package.

This cluster config is also a nice starter reference for adding load-balancing capabilities to external AWS instances. Given the volatile nature of VM image mappings, I’ve split the cluster config off into its own file for scripted generation.

In Summary

I’m very pleased with nginx. It has been very stable — the only time I’ve taken it down is when setting it up for infinte HTTP 302 redirects, and even then it took several hours of user activity to knock it over. The configuration syntax is very powerful, and I haven’t once been wistful for my old Apache habits :) .

The Beginning

Monday, January 12th, 2009

a good place to start.

i’ve been holding the CantRemember.com domain for years now, mostly so that i’d have awesome e-mail address. now, the time has come to do something with it. something is the operative term

this site exists to host my fun projects. i have some in mind, but i don’t like to make promises. the wiki is my central source of documentation, and this blog gives me a channel for constructive output. initial thanks go out to:

plus countless other blogs and threads. i’m now going to spend a few initial posts catching up on things that i learned while i was doing all this setup. hopefully it will be useful to others :)

sit back and relax. enjoy the show