I wrote this simple Node.js folder-based HTTP server that serves static and dynamic pages for domains I've bought but not yet used.
You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
Dave Winer 7dbbce413b v 0.7.10 6 years ago
docs v0.7.9 6 years ago
prefs v0.71 8 years ago
LICENSE Initial commit 9 years ago
README.md v 0.7.10 6 years ago
package.json v 0.7.10 6 years ago
pagepark.js v 0.7.10 6 years ago

README.md

PagePark

I wrote this simple HTTP server to park domains I've bought but not yet used. But I kept going and added all the features I want to help me manage my own websites, far beyond just parking them. But I liked the name so I kept it. Think of it as a nice park where you keep your pages. ;-)

It's written in JavaScript and runs in Node.js.

Each domain is in its own folder. The content for that domain is in the folder. I went a little wild with content types, it can serve Markdown docs, or run JS code. And of course HTML, text files, images, movies, etc.

Yet it's still very simple. Which is the point. ;-)

It's 90 percent of what all web servers do, so if you learn how to run PagePark, you're learning how to run a web server. A real one you can use to host your sites. And it's easy to hack the code if you want to.

How to

  1. Create a folder to host your website.

  2. Copy all the files from the downloaded folder into that folder.

  3. npm install

  4. node pagepark.js

Screen shot

Here's a screen shot of my PagePark server folder.

How it works

  1. PagePark will automatically create a prefs sub-folder and a domains sub-folder.

  2. Add your web content under domains. Each folder's name is the name of a domain. The contents within the folder are what we serve. Screen shot.

  3. Serves all major media types including audio and video. Files whose names end with .md are passed through the built-in Markdown processor. Files ending with .js are interpreted as scripts. The text they return is what we serve. Here's an example of a script that I have running on one of my servers.

  4. The prefs folder contains a file of settings you can change, prefs.json. These include the port that the server runs on and the name of the index file (see below).

  5. stats.json contains information generated by the server including the number of times the server has started, how many hits it's received (all time and today), and hits by domain.

  6. mdTemplate.txt is the template we use to serve Markdown text. You can edit this file to provide a common template for all your Markdown documents.

  7. If a request comes in for a folder, we scan the folder for a file whose name begins with index and serve the first one we find. So the index file can be HTML, Markdown or a script, or any other type PagePark can serve.

  8. If you want to run PagePark from a folder different from the one that contains the app, set the pageparkFolderPath environment variable to point to that folder.

  9. There are three special endpoints on all domains: /version, /now and /status that return the version of PagePark that's running, the time on the server and the stats and prefs.

File extensions

The extension of a file determines how PagePark serves it.

.txt

The text in the file is returned, the type is text/plain.

applejoke.txt

.xml

The text in the file is returned, the type is text/xml.

davetwitterfeed.xml

.json

The text in the file is returned, the type is application/json.

wodemo.json

.png

The contents of the file is returned with type image/png.

tree.png

.opml

The outline is rendered as an expandable outline, the type returned is text/html.

states.opml

.md

The text in the file is passed through a Markdown processor and that text is returned. The type returned is text/html.

test.md

.js

We run the script, and the return value is returned to the caller, with type of text/html. Here's the source code for the script in the demo below.

butt.js

Port 1339

The first time you run PagePark it will open on port 1339. You can change this by editing prefs.json in the prefs folder.

This means if you want to access a page on your site, the URL will be of the form:

http://myserver.com:1339/somepage.html

The normal port for HTTP is 80. That would have been the natural default, however a lot of Unix servers require the app to be running in supervisor mode in order for it to open on port 80. You can do this by launching PagePark this way:

sudo node pagepark.js

I made the default 1339 because I wanted it to work "out of the box" for first-time users.

Mapping port 80 to 1339

Here's a magic incantation that works on Ubuntu that maps requests for port 80 to port 1339.

sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 1339

Example pages

http://noderunner.org/ -- simple home page

http://lucky.wtf/ -- images

http://lucky.wtf/test.md -- markdown page

http://lucky.wtf/badass/ -- index file in a sub-directory

http://lucky.wtf/badass/butt.js -- a page implemented in a script

http://karass.co/nosuchfile.html -- file not found

http://noderunner.org/version -- the version of PagePark that's running on the server

http://noderunner.org/now -- the time on the server

http://lucky.wtf/transcend.opml -- a page written in OPML

http://lucky.wtf/transcend.opml?format=opml -- the OPML source of the page

config.json files in domains folders

On every request, PagePark looks at the top level of the domain folder for a file named config.json. If it finds it, it reads it and the values in the file control how the request is handled.

Here's a docs page that lists the values and what they control.

JavaScript sample code

I've iterated over the code to try to make it good sample code for JavaScript projects.

I wanted to make code that could be used for people who are just getting started with Node, to help make the process easier.

There will always be more work to do here. ;-)

Updates

v0.7.9 11/8/17 by DW

A new top-level config.json option, flUnicasePaths.

The feature is documented.

A blog post about the feature.

v0.7.8 10/3/17 by DW

New features supports logging over WebSockets. To enable, in config.json set flWebsocketEnabled true, and assign a port to the WS server in websocketPort. On every hit, PagePark will send back a JSON structure containing information about the request.

Note: There's a new dependency, nodejs-websocket, so when installing this update you have to do an npm install before running pagepark.js.

v0.7.7 9/26/17 by DW

Added two config.json options and changed the name of another.

  1. Changed the name of s3Path to fargoS3Path. It was only used in serving Fargo websites, on one of my servers. It's a very specific bit of functionality. Far more specific than the name implies.

  2. New meaning for s3Path option. If specified it points to an S3 location that PagePark will serve the site from. It should begin with a slash but not end with one. All requests for the domain are mapped to requests at the S3 location. Additionally, it processes the request through the same filters it uses for local files, so files ending with .md are passed through the Markdown processor and files that end with .opml are processed as outlines. It follows all the rules outlined on this page.

  3. New option: localPath, points to a directory on the local system where files are served from exactly as in #2 above.

A caveat on this release, I had trouble upgrading one of my servers with this version because of changes to a couple of packages we use. I was able to fix all the problems by installing the latest stable version of Node, as described on this page. If you have trouble, re-open this issue, and we can look into it.

v0.7.6 9/16/17 by DW

Two changes:

  1. The mime package had a breaking change. I fixed it so it no longer breaks PagePark. Thanks to Davis Shaver for the report.

  2. Changed the version number system to use the same x.y.z system used by other Node apps. This is v0.7.6.

v0.75 7/17/17 by DW

Check stats every minute instead of every second. This means that servers that take a lot of hits won't be writing the stats file so often. If you run PagePark out of a shared folder this can mean a lot of updating, it's not really protecting against anything critical.

v0.74 7/5/17 by DW

If a request comes in for domain.com/hello and it's a directory, redirect to domain.com/hello/.

v0.73 6/17/17 by DW

Replaced daveopml with opmltojs, a new package that builds on the xml2js package.

Rebuilt the OPML rendering code to use opmltojs. This allows the <head> info to be transmitted to the rendered page. And the full OPML structure is embedded in the page, meaning the page can render itself without calling back to the server to get missing bits from the OPML.

Added a new pref, flCacheTemplatesLocally, that allows you to say you don't want the OPML and Markdown templates cached. Add it to your prefs.json file. It defaults true, because that was the previous behavior (no breakage). This probably won't matter to people who aren't iterating over template development.

Made the URLs of the OPML and Markdown templates configurable through prefs.json. So if you want to do something nicer than I have, you can, without having to modify pagepark.js.

v0.72 6/7/17 by DW

Factored the local utils and opml modules, instead using the new daveutils and daveopml NPM packages.

We now look for prefs in config.json in in the same folder as pagepark.js, if it's not present, we look for prefs.json in the prefs folder. All my other server software uses config.json, so PagePark fits in better.

Improved rendering outlines, so that you can put an OPML file anywhere, and refer to it and it will render as an outline. If you add ?format=opml to the URL it will return the XML

v0.71 7/31/16 by DW

The repository now includes mdTemplate.txt in the prefs folder. Previously it would download this file from one of my sites the first time it was used. This is a more modern way to distribute it.

v0.70 12/8/15 by DW

It used to be that a file had to exist in order for you to redirect from it in config.redirects. Now it doesn't have to exist.

v0.68 8/31/15 by DW

Small change in error handling when we delegate a request. The previous method would cause PagePark to crash if the app we're trying to delegate to isn't running. Thanks to Dan MacTough for the help fixing this. ;-)

v0.67 7/30/15 by DW

New redirect feature for individual pages.

In config.json for the domain containing the file you want to redirect, create a struct called redirects. The name of each element is the path to a file in the folder, and the value of each is the URL we will redirect to. On each request for that domain, we look in the redirects table to see if the request should be redirected.

Here's an example of the config.json for smallpicture.com. It redirects from an old version of the outliner howto to the newer version.

v0.66 7/19/15 by DW

In prefs.json a new value, legalPathChars, defaults to the empty string. In this string you can specify characters that are legal in paths on your server. We are very conservative in what we will allow in paths, but if you need to use one of the characters that we consider illegal, add it to this string.

For example, I am redirecting urls from discuss.userland.com to a static archive. It was a Manila site, so it uses a in the URLs, a character which PagePark by default considers illegal. Here's an <a href="http://discuss.userland.com/msgReader$18647">example</a>. I set legalPathChars to "" in prefs.json, and it lets the character through, and it is redirected by the very clever little script that handles redirection for that domain.

v0.65 7/16/15 by DW

In prefs.json a new value, error404File, defaults to prefs/error.html

When there's an error, we read that file and send it back as the text of the 404 response.

The default value is more or less exactly the same text earlier versions of PagePark returned. It's actually in HTML, that's the difference. As opposed to being plain text.

v0.64 7/7/15 by DW

I wanted to do a site redirect that was more than just a domain name change.

If a request came in for: http://archive.scripting.com/2007/12

I wanted it to redirect to: http://scripting.com/2007/12.html

That was accomplished with a new element in config.json: jsSiteRedirect. Its value is a JavaScript expression. You can access the path in the variable parsedUrl.pathname.

This is what config.json in the folder archive.scripting.com looks like.

{"jsSiteRedirect": "'http://scripting.com' + parsedUrl.pathname + '.html'"}

v0.62 6/25/15 by DW

Code cleanup and factoring in opml.js.

v0.61 6/24/15 by DW

Files with the extension .opml are now rendered as outlines.

There's a template for outlines, it uses the new outlineBrowser toolkit. Here's an example of an OPML file rendered through the new template.

If you want the raw OPML text from the server, you can do it one of two ways:

  1. Add ?format=opml at the end of the url. Example.

  2. If you're making the request in a program, you can add an Accept header with the value: text/x-opml, following a convention established by the OPML Editor.

As with scripts and markdown files, you can turn the feature off in config.json.

v0.60 5/26/15 by DW

When delegating requests, pass redirects back to the client, don't follow them. This was necessary so that that OAuth dance with Twitter in nodeStorage would work.

There's a new structured listing of the source code of PagePark, linked to from the flat listing (above). This makes it easy to see how the code is organized. It gets pretty deeply nested! The outline view makes that manageable. I use the OPML Editor on a Mac to edit PagePark.

v0.59 5/23/15 by DW

You can now delegate requests to apps running on other ports on your server machine.

There's a new optional section of the prefs. json file where you specify these mappings. It's called domainMap. It's a set of name-values, where the name is the domain, and the value is the port the requests are mapped to.

Here's an example of a prefs.json that has a domainMap specified. It maps twitter.radio3.io to the process running on port 5342, twitter.happyfriends.camp to 5338, and any request on judgment.club to the process on port 5351.

The name matches the end of the HOST header for the request, so a request for judy.judgment.club will map, as will renee.judgment.club and judgment.club.

v0.57 5/11/15 by DW

PagePark has pre-defined pages, /now, /version and /status, whose values are returned by PagePark itself. It used to be that they took precedence, so if a site defines pages with those names, the internal ones would be served instead. Now we only serve them if the site didn't define it.

The urlSiteContents feature now transmits search params. It still will only forward GET calls. This needs to be updated in a future version.

PagePark now supports wildcards. Suppose you want to serve all the names from mydomain.org with a wildcard. Create a sub-folder of the domains folder with the name *.mydomain.org. If a request comes in for a sub-domain of mydomain.org that doesn't have its own folder, we'll route it through that folder. You can combine this feature with the urlSiteContents feature, or script-implemented pages.

We also set the X-Forwarded-Host and X-Forwarded-For headers on urlSiteContents requests.

v0.56 5/5/15 by DW

New prefs and config values that allow you to disable processing of scripts and Markdown files. By setting the values in prefs.json, you control all domains on the server. And by adding the values to config.json, in the folder the site is served from, you can turn them off selectively by site. I needed to turn off script processing for .js files served from River4, to make it possible to serve a full river from PagePark.

v0.55 4/26/15 by DW

With this release you can serve domains whose content is stored elsewhere on the web.

There's a new optional element of config.json, urlSiteContents. Its value is the URL of a directory on the web that stores the content of the domain.

As an experiment, I mapped the domain my.this.how to point to a folder in my public Dropbox folder. Here's how I did it.

  1. First I created a new sub-folder of my public Dropbox folder called pageParkDemo. I put one file in that folder, index.html.

  2. I used the web interface of my domain registrar to point my.this.how at the IP address of my PagePark server.

  3. On my PagePark server, I created a sub-folder of the domains folder called my.this.how.

  4. I created a config.json file in that folder, pointing to the pageParkDemo folder in my Dropbox folder.

To test the setup, I just went to my.this.how, where I saw "this is just a demo".

Just for fun I put a picture in that folder, to see if it works.

Obviously it can get a lot more elaborate, and you can store the content anywhere, not just in a Dropbox folder. The key is that the content be accessible over the web.

v0.54 2/18/15 by DW

A new feature for pages implemented as scripts. If the script returns the value undefined PagePark will not return a value to the HTTP client, it assumes that the script will do this.

To make it possible for a script page to return a value to the client, there's a new built-in function httpReturn. It takes two parameters, the value, a string, and the type, a MIME type. For example you might have a script that makes an HTTP request and returns a value based on the result to the caller.

Here's an example script that illustrates.

v0.51 1/18/15 by DW

Created utils.js in the lib folder, and require it in pagepark.js.

New feature: If there's a file called config.json in a domain folder, we read it on every request, and values in that file can change the behavior of the server. The first feature allows you to do a whole-site redirect. Useful if you want to have several names map to the same content. Here's an example of the config.json file that maps a domain to nodestorage.io.

v0.48 1/8/15 by DW

The default port the server boots up on is now 1339. Previously it was 80, which is the standard port for HTTP, but on many OSes this requires PagePark to be running in supervisor mode. I added docs above to explain this.

Changed package.json so that only request and marked were listed as dependencies. Apparently the others are included in Node without having to list them.

Instead of keeping our own MIME type table, we use the Node mime package, which is also included as a dependency in the package.json file.

v0.47 1/7/15 by DW

Added a package.json file to the repository.

Fixed first-time startup problem creating prefs.json and stats.json.

Also, we now make sure the domains folder exists at startup.

Fixed a problem in handling requests if you specified a different folder for PagePark to serve from.

Questions, comments?

Please post a note on the Server Snacks mail list or post an issue here.