Setting up a home servers for developers, Part 3: SSH

One of the first things to set up is SSH access.  SSH allows you to open a remote prompt without physically being at the computer.

In this case, we’re going to use OpenSSH.  OpenSSH was at one point developed for OpenBSD, but is now available as a port for many operating systems.  In this case, we want the portable version.  It is downloadable from ftp://ftp.openbsd.org/pub/OpenBSD/OpenSSH/portable/

Download this and unzip it, then compile and install it as normal; the only difference being the prefix (the directory under which the software is to be installed):

cd openssh-5.1p1

./configure --prefix=/opt/openssh-5.1p1

make

sudo make install

The SSH daemon, sshd, is now installed and runnable:

/opt/openssh-5.1p1/sbin/sshd

This is nice, but we want the daemon to start with the machine.  To do this, we create a new event for Upstart.  You can read more about upstart here, but for now we just need to create a new configuration file which describes our service.

/etc/events.d/sshd

start on runlevel 2
stop on runlevel [!2]

exec /opt/openssh-5.1p1/sbin/sshd

This starts the daemon when the system enters into runlevel 2 (system initialization is complete, networking is available, and the user is at the command prompt/graphical interface).  For this app, that is all that is needed.

Now, if you open up an SSH client from another machine, such as PuTTY, you should be able to connect successfully.

Uncategorized

Comments (0)

Permalink

Setting up a home server for developers, Part 2: Getting Online

Once the base system has been set up, the next step is to get online.  I’m using a wireless connection; probably not the best idea for a server, but it’s what’s there.

The most important command when setting up wireless is iwconfig.  This command is part of the Wireless Tools for Linux package, and allows a wireless connection to be configured from the command line.  For setting up a WEP connection, most likely only two commands are needed:

> sudo iwconfig wlan0 key 1B0C8EFD302
> sudo iwconfig wlan0 essid LinkSys

It is important to put the key in before the essid, since putting the essid will cause the connection to occur.  If you’re lucky, this should be enough to get online:


> ping www.google.com
PING www.l.google.com (209.85.165.103) 56(84) bytes of data.
64 bytes from eo-in-f103.google.com (209.85.165.103): icmp_seq=1 ttl=252 time=26.4 ms

Once you have it working from the command-line, it needs to be configured permanently.  This is done by editing /etc/network/interfaces.  It looks something like this:


# Wireless
auto wlan0
iface wlan0 inet dhcp
gateway 192.168.1.1
netmask 255.255.255.0
wireless-key 18012C3E4D
wireless-essid LA5Q1

The first line causes the interface to come up automatically on server boot.  The second sets it up as an internet network adapter which uses dhcp.  Even though I plan on using  a static IP, I set it up to use DHCP here so that it could set up DNS, etc. correctly.  The following lines are sent on to ifconfig.  The ones that begin with wireless- are sent to iwconfig.  So the wireless-key line is the same as iwconfig wlan0 key …

Reboot, and you should be online from the start.

The configuration above uses a dynamic IP, whereas I want it to be static so that I always know how to get to the machine.  Fortunately, my router has an option to map MAC addresses to a specific IP, as most recent ones do.  Set that up, and you should be good to go.

Uncategorized

Comments (0)

Permalink

Setting up a home server for developers, Part 1: Introduction

I’ve started working on converting one of the computers at our house into a server for home.  I figured I should document the process in case anyone else wants to do the same, or just so I can keep track of what I’ve done.

So, what is this server for?  As the title suggests, I’m gearing my configuration towards what a software developer would use a server for.  However, plopping a server on your network has advantages for even a home user:

  • By setting up a network share and consolidating photos, music, videos, etc onto that drive, they are accessible from anywhere in the house and are easily backed up.
  • You can stream out music, videos, etc and provide more storage than you would have on an individual desktop/laptop
  • It can act as a print server
  • If you wanted it could act as an advanced firewall, proxy, etc

In addition, there are several services you can set up as a developer that go above and beyond these:

  • Version control
  • Bug tracking
  • A running LAMP/JEE server
  • Continuous integration builds
  • A Maven repository + proxy
  • A Wiki
  • Other server-side apps that may come along

These may sound like overkill for an individual developer, but if you are going to be working on a variety of side-projects, the become critical.  Here’s some example use cases:

  • You create a java .jar file containing some useful functions.  You check the code in, kicking off a continuous integration build.  This build performs a Maven install, which installs it into the local repository.  The next time you want to use this library, all you have to do is declare it as a dependency and let Maven pull it out of your repository
  • You are working on a project, and know that every time you want to create a new button (say), there is a list of 5 files that have to be updated.  Create a wiki page on the internal wiki and put that information there, since it is not useful to others.
  • You decide to take a break from a project for a while.  By using a bug tracker as at least a simple to-do list, you can quickly come back up to speed when you return to it.

The benefits may not be huge, but it helps keeps things organized a little better.

I chose to run Ubuntu Linux on the server.  Linux is free, which is always nice, and Ubuntu seems to be the most popular distribution at the moment.  If you’re not familiar with Ubuntu, it’s essentially a modified version of Debian.  Debian is known for being slow to update, which made it popular for a server machine because it was generally very solid.  Ubuntu took that as a starting point, but has been much quicker at integrating and updating to the latest stuff.

Ubuntu has a server edition, which is what I used for this.  Just burn the installer to a CD, and off you go.  When you run the installer, it attempts to connect to the network to install some additional software.  However, the installer live cd didn’t hook up to my wireless on its own.  This turned out to be a good thing, I think.  Instead, it just installed a very bare base system with little on top.

My goal when setting this server up is to separate out the base system and the various server software that’s running on top of it.  The base system and libraries come from apt-get and are in the usual /usr/bin, etc spots.

The actual server software, however, I’ve opted to do a bit differently.  Rather than simply install the preconfigured packages, I’m downloading the software and installing it myself.  The software will go into /opt/<package-name>, with bin, lib, etc libraries under there.   This may be a little bit of overkill, but my reasoning is:

  • /opt is its own partition, so it can be backed up/restored easily (as is /home)
  • I find it a little less overwhelming to install and configure the packages one at a time, making sure I know what I’m doing at each step
  • Maybe it’s just me, but I hate having all binaries in /usr/bin, etc.  It just seems too messy.  I much prefer having a folder for each package.
  • I’m a masochist.

So, I went through the minimal install, and when presented with a list of which servers I wanted to install, chose none.  A few minutes later, I had a working command line.

Part two I’ll talk a little bit about how to set up wireless networking and get online.

Uncategorized

Comments (1)

Permalink

Loading Java .properties Files from PHP

Java typically stores its configuration in files that end with a .properties extension.  These files are essentially key-value pairs of data.  Java SE has a built-in class, java.util.Properties, for reading and writing these files.

Sometimes, though, you may want to read these files from other languages.  For example, database information is often stored in a .properties file.  If another application wishes to connect to the same database, it is better to read that file rather than create a copy.  That way, if the password changes (for example), you only have one place to update.

The basic syntax of the file is easy to understand.  It is line-based.  Each line is either a blank line, a comment line, or a key-value pair.  Keys are separated from values by an =, :, or space.  Easy, right?

Well, there are a few complications.  You can read the full spec here, but this is a brief summary of some of the issues with creating a fully-compliant reader/writer:

  • An odd number of backslashes indicates that the next line is a continuation of the previous line, unless the line is a comment line, which can’t be continued
  • Comments are started with # or !, and extend to the end of the line; however, if a # or ! appears after the first non-whitespace character, they are not considered to be comments.  When writing, though, these values are still escaped using a backslash.
  • =, :, or space separate a key from a value.  However, these values can be escaped so that they can be used as part of a key or value string.

So I’ve got a working PHP port of the java.util.Properties class.  Most of the common methods are implemented with the same name as their Java counterpart.  The header documentation of the class goes into more details on what is and is not implemented, and where to find the corresponding functionality.

View the full source below, or you can download a .zip file from here containing the .php file and the tests showing its correctness.

NOTE: If you copy and replace from the source below, you must replace all instances of f_write with fwrite. This is due to security restrictions with posting PHP code here.

/**
 * PHP Version of the class java.util.Properties.  Loads and saves Java
 * .properties files.
 *
 * The methods inherited from Hashtable are not implemented, as they are
 * merely another interface to the same functionality.  The save method is not
 * implemented, as it is considered deprecated; use store instead.
 *
 * In general, methods which are overloaded to accept an optional parameter
 * are combined into a single method with a default value of null for the
 * optional parameters.  Methods which take versions of streams, writers,
 * etc. are consolidated into a single method taking a PHP stream.
 *
 * The list() methods are replaced with debugList(), as list is a reserved
 * word in PHP.
 *
 * For more information on the .properties format, see
 *   http://java.sun.com/javase/6/docs/api/java/util/Properties.html
 */
class Properties
{
	/**
	 * Constructor.  Takes an optional Properties object defining the default
	 * values of the properties.
	 */
	public function __construct($defaults = null) {
		if($defaults != null) {
			$this->defaults = $defaults;
		}

		$this->properties = array();
	}

	/**
	 * Retrieves a property value.  If a value is not found for this key, the
	 * defaults object is searched for a default.  If not default value is
	 * found there, the default value passed into this function is used.  If
	 * a value still cannot be determined, null is returned.
	 */
	public function getProperty($key, $defaultValue = null) {
		$value = null;

		if(array_key_exists($key, $this->properties)) {
			$value = $this->properties[$key];
		}

		if($value == null && $this->defaults != null) {
			$value = $this->defaults->getProperty($key);
		}

		if($value == null) {
			$value = $defaultValue;
		}

		return $value;
	}

	/**
	 * Sets a property value.  Returns the old value, if available.  No defaults
	 * are used when searching for the old value; a value is returned only if it
	 * was explicitly set.
	 */
	public function setProperty($key, $value) {
		$oldValue = null;
		if(array_key_exists($key, $this->properties)) {
			$oldValue = $this->properties[$key];
		}
		$this->properties[$key] = $value;
		return $oldValue;
	}

	/**
	 * Returns the names of all keys in this property set, including keys in the
	 * default property set if no value is defined in this property set.
	 */
	public function propertyNames() {
		$keys = array();
		if($this->defaults != null) {
			$keys = $this->defaults->propertyNames();
		}

		$myKeys = array_keys($this->properties);
		sort($myKeys);
		for($i = 0; $i < count($myKeys); $i++) {
			if(!array_key_exists($myKeys[$i], $keys)) {
				array_push($keys, $myKeys[$i]);
			}
		}

		return $keys;
	}

	/**
 	 * Writes the properties to the given stream, in the format defined by
	 * java.util.Properties.
	 */
	public function store($handle, $comment = null) {
		if($comment != null) {
			f_write($handle, "#" . $this->escapeComment($comment) . PHP_EOL);
		}
		f_write($handle, "#" . date("D M d H:i:s T Y") . PHP_EOL);

		$propertyNames = array_keys($this->properties);
		for($i = 0; $i < count($propertyNames); $i++) {
			f_write($handle, $this->escapeKey($propertyNames[$i]) . "=" . $this->escapeValue($this->properties[$propertyNames[$i]]) . PHP_EOL);
		}
	}

	/**
	 * Loads a file from the given stream, in the format defined by
	 * java.util.Properties
	 */
	public function load($handle) {
		$this->properties = array();

		$logicalLine = "";
		$appendNext = false;
		while(!feof($handle)) {
			$line = fgets($handle);
			$line = rtrim($line, "\r\n"); // Chomp!
			$line = ltrim($line);

			// If the previous line was a continuation line, append this one
			if($appendNext) {
				$logicalLine .= $line;
				$appendNext = false;
			} else {
				$logicalLine = $line;
			}

			// Is it a comment line?
			// Check this before checking for continuations, as comments cannot be continued
			if(preg_match("/^[ 	]*[#!]/", $logicalLine)) {
				continue;
			}

			// Is this line continued?
			// A continuation line is one ending in an odd number of backslashes
			$numTrailingBackslashes = 0;
			while((strlen($logicalLine) - $numTrailingBackslashes - 1) > -1 &&
				  $logicalLine[strlen($logicalLine) - $numTrailingBackslashes - 1] == "\\") {
				$numTrailingBackslashes++;
			}
			if($numTrailingBackslashes % 2 == 1) {
				// If it is a continuation line, append a newline.. this will
				// removed when the line is unescaped, but appending it in
				// advance prevents backslashes other than the final one from
				// escaping the next line, e.g. in the case
				//   a=b\\\
				//   n
				// If you are not careful, this will be transformed into
				//   a=b\\n
				// And bad things could happen.
				$logicalLine .= "\n";
				$appendNext = true;
				continue;
			}

			// Now that we've combined the physical lines to form one large
			// logical line, see if it's empty.
			if(strcmp($logicalLine, "") == 0) {
				continue;
			}

			// Split into key-value pairs
			$matches = array();
			$key = "";
			$value = "";

			// This regex splits the string into 3 sections:
			//   - Leading whitespace plus all non-whitespace characters up to
			//     the first unescaped :, =, space, or tab character
			//   - The :, =, space, or tab character itself, which separates
			//     the key from the value
			//   - The rest of the string
			// If this match succeeds, the first and third pieces form the key
			// and value. Otherwise, the entire string is considered to be a
			// key with no value.
			if(preg_match("/^([ 	]*.*?[^\\\\])([:= 	]+?)([^:= 	].*)$/s", $logicalLine, $matches)) {
				$key = $matches[1];
				$value = $matches[3];
			} else {
				$key = $logicalLine;
			}

			$key = $this->unescapeGeneric($key);
			$value = $this->unescapeGeneric($value);

			$this->properties[$key] = $value;
		}

	}

	/**
	 * List the contents to an output stream.  This is useful for debugging.
	 */
	public function debugList($handle) {
		f_write($handle, "-- listing properties --" . PHP_EOL);
		$propertyNames = $this->propertyNames();
		for($i = 0; $i < count($propertyNames); $i++) {
			f_write($handle, $propertyNames[$i] . "=" . $this->getProperty($propertyNames[$i]) . PHP_EOL);
		}
	}

	private function escapeKey($key) {
		return str_replace(" ", "\\ ", $this->escapeGeneric($key));
	}

	private function escapeValue($value) {
		$value = $this->escapeGeneric($value);

		// Values have leading space replaced, but none other
		if(strlen($value) > 0 && strcmp($value[0], ' ') == 0) {
			$value = "\\" . $value;
		}

		return $value;
	}

	private function escapeGeneric($value) {
		$value = str_replace("#", "\\#", $value);
		$value = str_replace("!", "\\!", $value);
		$value = str_replace("=", "\\=", $value);
		$value = str_replace(":", "\\:", $value);
		$value = str_replace("\t", "\\t", $value);
		$value = str_replace("\n", "\\n", $value);
		$value = str_replace("\r", "\\r", $value);
		return $value;
	}

	private function escapeComment($comment) {
		return preg_replace("/\n([^#])/", "\n#\\1", $comment);
	}

	private function unescapeGeneric($value) {
		$value = preg_replace("/\\\\([^\\\\trn\\n])/s", "\\1", $value);
		$value = str_replace("\\\n", "", $value);
		$value = str_replace("\\t", "\t", $value);
		$value = str_replace("\\r", "\r", $value);
		$value = str_replace("\\n", "\n", $value);
		return $value;
	}

	private $properties /* Array */;
	private $defaults /* Properties */;
}

Uncategorized

Comments (1)

Permalink

ChaCha For a Day

Today is the semi-official Day Without Google.  I say semi-official, because it is sponsored by the Alt Search Engines list, which is obviously a bit of a biased site.

While looking at their list of top 100 alternative search engines (which is surprisingly difficult to find, given it is the focus of their site) I learned of the search engine ChaCha.

ChaCha provides a normal search, similar to Google.  However, they also have what they call a “guided search”.  Enter a question and press the Guided Search button.  A chat window opens, and a human researches your question, and provides an answer.

Of course, this takes longer than plugging a term into Google, but it is an interesting alternative for people who aren’t savvy enough to sift through a set of search results to find the information they need (think your mom).

What really makes this interesting, though, is that you can do a free search through SMS from your cell phone.  Simply text 242242 (ChaCha… even easy to remember) with your question, and get an answer texted back.  This can be a great option for getting answers on the go, when you’re not near a computer.

So how does it work?  Answerers are volunteers who are paid for each successful answer they give.  The pay is $0.20/answer, and you can log in and provide answers whenever you want.  Once you become efficient, it is expected that you will make $4-$6 an hour.  After going live, ChaCha quickly gained 20,000 guides, although there is no set schedule, so there is no guarantee how many guides will be online at a time.

I’ve tried it a couple of times with mixed results.  Usually you can Google the answer more quickly yourself, but for an inexperienced user or someone on the go, ChaCha could be just the thing.

Uncategorized

Comments (0)

Permalink

Mysterious Number Transmissions

If you’re even the most casual fan of Lost, you’ve heard of the infamous numbers.  These numbers were picked up as a mysterious transmission emanating from the island.

What you may not realize is that mysterious number transmissions are an actual phenomenon.  Mysterious transmissions containing a human or synthesized voice reading a stream of numbers are routinely picked up around the world on shortwave radio sets.

The numbers may be transmitted at a regular schedule or intermittently.  Some are broadcast on a single frequency, some are simultaneous transmissions available on two or more frequencies.  The numbers usually have some sort of identifier before and after each transmission.  Stations are informally named after these intros; for example, one of the most famous stations is called the Lincolnshire Poacher, because it starts with the bars from the traditional song with the same name.

So what is the source of these mysterious transmissions?  Are they mysterious pockets of space-time, waiting to crash-land us far from civilization?

Not quite.  An important clue was uncovered in Miami, FL, in 1998.  Home to a large number of Cuban exiles, Miami is also home to a group known as Brothers to the Rescue.  Brothers to the Rescue is an activist group made up of Cuban exiles who oppose the Cuban government, and Fidel Castro in particular.

The Cuban Five, a group of five spies working for the Cuban government, traveled to Miami using fake papers and infiltrated the group.  They then fed information back to the Cuban government, which eventually led to the downing of two civilian aircraft (it is worth noting that there is a large movement to free the Cuban Five; the justification is that the Miami-based exile groups were sponsoring terrorism against Cuba, and the five were there to monitor these groups).

The interesting part happened three years before their arrest, however.  An FBI agent entered the men’s apartment, and copied from his laptop a decryption program.  The FBI then used this program to decrypt the numbers transmitted from Attencion, one of the mysterious number stations.  One of the messages decoded contained the words:

Under no circumstances should [agents] German nor Castor fly with BTTR or another organization on days 24, 25, 26, and 27

Particular damning since planes were shot down.

So the number stations, or at least this particular number station, seem to be transmissions from governments to agents in the field, or other individuals they cannot contact directly.  This was also confirmed by a UK Department of Trade and Industry Spokesperson:

These [numbers stations] are what you suppose they are. People shouldn’t be mystified by them. They are not for, shall we say, public consumption.

It is believed that the numbers are encoded using what’s known as a one-time pad.  A stream of random numbers is generated, and then each character is assigned a number.  The two numbers are added together for one result.  For example:

A     B     C
1     2     3
13823 32423 34289
13824 32425 34292

Without access to the set of random numbers, the code is literally unbreakable.  The only way it could be broken is for the same set of numbers to be reused, which has happened before.

Many people find these transmissions mesmerizing, mysterious numbers broadcast into the ether.  In 1997, a 4-CD set was published containing a compilation of recordings from number stations.  The CD’s can be legally downloaded here.
So the mystery would seem to be solved.  However, there are still a few things left to question:

  • Why do all these governments, all over the world, use the same means of transmission to send information to their agents?
  • Why aren’t these stations sabotaged or jammed, to prevent dissemination of information?
  • Although these stations are broadcast in many languages, there is no French station.

And of course, it’s always possible that one of those transmissions is coming from a small island in the middle of the Pacific….

Uncategorized

Comments (0)

Permalink

ResizeWindow is MoveWindow in Win32

If you’re looking for a Win32 method such as ResizeWindow, don’t strain yourself, it doesn’t exist.  Use MoveWindow instead, which allows you to set the window position and size.

Uncategorized

Comments (0)

Permalink

Where Do Stocks Exist?

In today’s world of Roth IRA’s and 401k’s, it’s easy to think of stocks as simply numbers in a bank statement. Stocks are more than that, however. Stocks are an actual part of ownership in a company. And, just as you can get cash you can hold in your hands from an electronic bank account, you can get a stock certificate you can hold in your hands.

To understand how this works, first it helps to understand the types of stock ownership.  As explained here, there are three ways the stock can be owned:

  • Street Name Registration.  Here, the broker actually buys, sells, and owns the stock, in your name.  If you want to be able to perform a limit order or any other type of special trade, it must have street name registration, since only brokers may place those types of orders.
  • Direct Registration.  You can own directly registered stock by buying and selling through the company itself, rather than through a brokerage.  Many large companies have “investor relations” pages that give you current investor information and let you buy directly.  For example, if you were interested in owning part of Coca-Cola, you can go to their shareholder’s information page.  From here, you are directed to use ComputerShare as a means to buy a Direct Registration stock (In fact, ComputerShare lists 136 companies that they manage the stock for, so if you want to buy Direct Registered stock, this would be a good place to start)..  Although you still do not own a physical certificate using this method, you are registered with the company as a stockholder, and receive information such as their annual report and the ability to vote in person or by proxy at the annual shareholder’s meeting.
  • Physical Certificate.  This is the stock equivalent of having cash in your hands.  You hold the actual certificate representing part ownership in the company.

There are two types of physical certificates: bearer and registered.  Bearer certificates work literally like cash: the bearer of the certificate is the owner of the stock.  The physical certificate must be relinquished in order to sell or transfer it.

More common is the registered stock.  This certificate contains on the front of it the name of the owner of the stock, similar to a check.  The stock can be transferred using a facsimile of the back, with the owner endorsing it to someone else.

So how do you get a paper certificate?  The easiest way is probably to buy a stock via Direct Registration and request one.  If you buy stock through a broker, you may be able to get one from them for a fee, usually in the $50-$100 range.

Another option is a company called OneShare.  OneShare gets a stock certificate representing one share in a company and frames it for you.

Finally, you can try EBay.  Be cautious, though, if buying a stock certificate for the ownership value; if it is a registered certificate without the proper endorsements, you will not be able to sell it.

So why would you want to own a piece of paper?  One reason would be if you wanted something physical you could keep in a safety deposit box/safe in case of emergency. It is worth noting that if you do intend to sell later, you want to be very careful with your certificate.  If it is lost, and is a registered stock certificate, a new one may be re-issued to the registrant.  However, in this case, an indemnity bond must be purchased.  This is an insurance policy on the stock, usually costing ~2% of the stock value.  In the case that the original certificate is redeemed erroneously, this policy covers the money lost by the company.

In any case, the most common reason for wanting to own a physical stock certificate is as a collectible or conversation piece.  In fact, there’s a whole field of stock certificate collection known as Scripophily (Scrip = “Ownership Right”, ophily = “Love of”).

Scripophily mainly focuses on rare/antique stock certificates, especially those of bankrupt companies.  However, you can easily think of all kinds of stock to own: your current company, Google, large companies such as Coca-Cola, plus all the bankrupt or near-bankrupt ones, such as Enron, Bear-Stearns, etc.

Uncategorized

Comments (0)

Permalink