I wrote this simple HTTP server to park domains I've bought but not yet used. But I kept going and added all the features I want to help me manage my own websites, far beyond just parking them. But I liked the name so I kept it. Think of it as a nice park where you keep your pages. ;-)
Each domain is in its own folder. The content for that domain is in the folder. I went a little wild with content types, it can serve Markdown docs, or run JS code. And of course HTML, text files, images, movies, etc.
It's 90 percent of what all web servers do, so if you learn how to run PagePark, you're learning how to run a web server. A real one you can use to host your sites. And it's easy to hack the code if you want to.
2. Add your web content under domains. Each folder's name is the name of a domain. The contents within the folder are what we serve. <ahref="http://scripting.com/2015/01/04/pageParkFolderScreenShot.png">Screen shot</a>.
3. Serves all major media types including audio and video. Files whose names end with .md are passed through the built-in Markdown processor. Files ending with .js are interpreted as scripts. The text they return is what we serve. Here's an <ahref="https://gist.github.com/scripting/2b5e237a105b6cb96287">example</a> of a script that I have <ahref="http://lucky.wtf/badass/butt.js">running</a> on one of my servers.
4. The prefs folder contains a file of settings you can change, prefs.json. These include the port that the server runs on and the name of the index file (see below).
5. stats.json contains information generated by the server including the number of times the server has started, how many hits it's received (all time and today), and hits by domain.
7. If a request comes in for a folder, we scan the folder for a file whose name begins with *index* and serve the first one we find. So the index file can be HTML, Markdown or a script, or any other type PagePark can serve.
8. If you want to run PagePark from a folder different from the one that contains the app, set the *pageparkFolderPath* environment variable to point to that folder.
9. There are three special endpoints on all domains: /version, /now and /status that return the version of PagePark that's running, the time on the server and the stats and prefs.
The text in the file is passed through a Markdown processor and that text is returned. The type returned is text/html.
</td>
<td>
<ahref="http://lucky.wtf/test.md">test.md</a>
</td>
</tr>
<tr>
<td>
.js
</td>
<td>
We run the script, and the return value is returned to the caller, with type of text/html. Here's the <ahref="https://gist.github.com/scripting/fd9e6720834958130f0a3d53b9f8dd15">source code</a> for the script in the demo below.
The first time you run PagePark it will open on port 1339. You can change this by editing prefs.json in the prefs folder.
This means if you want to access a page on your site, the URL will be of the form:
http://myserver.com:1339/somepage.html
The normal port for HTTP is 80. That would have been the natural default, however a lot of Unix servers require the app to be running in supervisor mode in order for it to open on port 80. You can do this by launching PagePark this way:
sudo node pagepark.js
I made the default 1339 because I wanted it to work "out of the box" for first-time users.
On every request, PagePark looks at the top level of the domain folder for a file named config.json. If it finds it, it reads it and the values in the file control how the request is handled.
Here's a <ahref="https://github.com/scripting/pagePark/blob/master/docs/config.md">docs page</a> that lists the values and what they control.
New features supports logging over WebSockets. To enable, in config.json set <i>flWebsocketEnabled</i> true, and assign a port to the WS server in <i>websocketPort.</i> On every hit, PagePark will send back a JSON structure containing information about the request.
Note: There's a new dependency, <ahref="https://github.com/websockets/ws">nodejs-websocket</a>, so when installing this update you have to do an `npm install` before running pagepark.js.
Added two config.json options and changed the name of another.
1. Changed the name of <i>s3Path</i> to <i>fargoS3Path.</i> It was only used in serving Fargo websites, on one of my servers. It's a very specific bit of functionality. Far more specific than the name implies.
2. New meaning for <i>s3Path</i> option. If specified it points to an S3 location that PagePark will serve the site from. It should begin with a slash but not end with one. All requests for the domain are mapped to requests at the S3 location. Additionally, it processes the request through the same filters it uses for local files, so files ending with .md are passed through the Markdown processor and files that end with .opml are processed as outlines. It follows all the rules outlined on <ahref="https://github.com/scripting/pagePark/blob/master/docs/config.md">this page</a>.
3. New option: <i>localPath,</i> points to a directory on the local system where files are served from exactly as in #2 above.
A caveat on this release, I had trouble upgrading one of my servers with this version because of changes to a couple of packages we use. I was able to fix all the problems by installing the latest stable version of Node, as described on <ahref="https://nodesource.com/blog/installing-node-js-tutorial-ubuntu/">this page</a>. If you have trouble, re-open <ahref="https://github.com/scripting/pagePark/issues/8">this issue</a>, and we can look into it.
1. The <ahref="https://www.npmjs.com/package/mime">mime package</a> had a breaking change. I fixed it so it no longer breaks PagePark. Thanks to Davis Shaver for the <ahref="https://github.com/scripting/pagePark/issues/7">report</a>.
2. Changed the version number system to use the same x.y.z system used by other Node apps. This is v0.7.6.
Check stats every minute instead of every second. This means that servers that take a lot of hits won't be writing the stats file so often. If you run PagePark out of a shared folder this can mean a lot of updating, it's not really protecting against anything critical.
Replaced <ahref="https://www.npmjs.com/package/daveopml">daveopml</a> with <ahref="https://www.npmjs.com/package/opmltojs">opmltojs</a>, a new package that builds on the <ahref="https://github.com/Leonidas-from-XIV/node-xml2js">xml2js</a> package.
Rebuilt the OPML rendering code to use opmltojs. This allows the <head> info to be transmitted to the rendered page. And the full OPML structure is embedded in the page, meaning the page can render itself without calling back to the server to get missing bits from the OPML.
Added a new pref, <i>flCacheTemplatesLocally,</i> that allows you to say you don't want the OPML and Markdown templates cached. Add it to your prefs.json file. It defaults true, because that was the previous behavior (no breakage). This probably won't matter to people who aren't iterating over template development.
Made the URLs of the OPML and Markdown templates configurable through prefs.json. So if you want to do something nicer than I have, you can, without having to modify pagepark.js.
Factored the local <i>utils</i> and <i>opml</i> modules, instead using the new <ahref="https://github.com/scripting/utils">daveutils</a> and <ahref="https://github.com/scripting/opml">daveopml</a> NPM packages.
We now look for prefs in config.json in in the same folder as pagepark.js, if it's not present, we look for prefs.json in the prefs folder. All my other server software uses config.json, so PagePark fits in better.
Improved rendering outlines, so that you can put an OPML file anywhere, and refer to it and it will render as an outline. If you add ?format=opml to the URL it will return the XML
The repository now includes <ahref="https://github.com/scripting/pagePark/blob/master/prefs/mdTemplate.txt">mdTemplate.txt</a> in the prefs folder. Previously it would download this file from one of my sites the first time it was used. This is a more modern way to distribute it.
It used to be that a file had to exist in order for you to redirect from it in <ahref="https://github.com/scripting/pagepark#v067-73015-by-dw">config.redirects</a>. Now it doesn't have to exist.
Small change in error handling when we delegate a request. The previous method would cause PagePark to crash if the app we're trying to delegate to isn't running. Thanks to Dan MacTough for the help fixing this. ;-)
In config.json for the domain containing the file you want to redirect, create a struct called <i>redirects.</i> The name of each element is the path to a file in the folder, and the value of each is the URL we will redirect to. On each request for that domain, we look in the redirects table to see if the request should be redirected.
Here's an <ahref="https://gist.github.com/scripting/8295f373c61dd9f5ce97">example</a> of the config.json for smallpicture.com. It redirects from an <ahref="http://smallpicture.com/outlinerHowto.html">old version</a> of the outliner howto to the newer version.
In prefs.json a new value, legalPathChars, defaults to the empty string. In this string you can specify characters that are legal in paths on your server. We are very conservative in what we will allow in paths, but if you need to use one of the characters that we consider illegal, add it to this string.
For example, I am redirecting urls from discuss.userland.com to a static archive. It was a Manila site, so it uses a $ in the URLs, a character which PagePark by default considers illegal. Here's an <ahref="http://discuss.userland.com/msgReader$18647">example</a>. I set legalPathChars to "$" in prefs.json, and it lets the character through, and it is redirected by the very clever little script that handles redirection for that domain.
In prefs.json a new value, error404File, defaults to prefs/error.html
When there's an error, we read that file and send it back as the text of the 404 response.
The default value is more or less exactly the same text earlier versions of PagePark returned. It's actually in HTML, that's the difference. As opposed to being plain text.
That was accomplished with a new element in config.json: jsSiteRedirect. Its value is a JavaScript expression. You can access the path in the variable parsedUrl.pathname.
This is what config.json in the folder archive.scripting.com looks like.
Files with the extension .opml are now rendered as outlines.
There's a <ahref="http://fargo.io/code/pagepark/defaultopmltemplate.txt">template</a> for outlines, it uses the new <ahref="https://github.com/scripting/outlineBrowser">outlineBrowser</a> toolkit. Here's an <ahref="http://montauk.scripting.com/test.opml">example</a> of an OPML file rendered through the new template.
If you want the raw OPML text from the server, you can do it one of two ways:
1. Add ?format=opml at the end of the url. <ahref="http://montauk.scripting.com/test.opml?format=opml">Example</a>.
2. If you're making the request in a program, you can add an Accept header with the value: text/x-opml, following a convention established by the OPML Editor.
As with scripts and markdown files, you can turn the feature off in config.json.
When delegating requests, pass redirects back to the client, don't follow them. This was necessary so that that OAuth dance with Twitter in nodeStorage would work.
There's a new <ahref="http://scripting.com/listings/pagepark.html">structured listing</a> of the source code of PagePark, linked to from the <ahref="https://github.com/scripting/pagepark/blob/master/pagepark.js">flat listing</a> (above). This makes it easy to see how the code is organized. It gets pretty deeply nested! The outline view makes that manageable. I use the OPML Editor on a Mac to edit PagePark.
You can now delegate requests to apps running on other ports on your server machine.
There's a new optional section of the prefs. json file where you specify these mappings. It's called domainMap. It's a set of name-values, where the name is the domain, and the value is the port the requests are mapped to.
Here's an <ahref="https://gist.github.com/scripting/af9df0198db5daa3c982">example</a> of a prefs.json that has a domainMap specified. It maps twitter.radio3.io to the process running on port 5342, twitter.happyfriends.camp to 5338, and any request on judgment.club to the process on port 5351.
The name matches the end of the HOST header for the request, so a request for judy.judgment.club will map, as will renee.judgment.club and judgment.club.
PagePark has pre-defined pages, /now, /version and /status, whose values are returned by PagePark itself. It used to be that they took precedence, so if a site defines pages with those names, the internal ones would be served instead. Now we only serve them if the site didn't define it.
The <i>urlSiteContents</i> feature now transmits search params. It still will only forward GET calls. This needs to be updated in a future version.
PagePark now supports wildcards. Suppose you want to serve all the names from mydomain.org with a wildcard. Create a sub-folder of the domains folder with the name *.mydomain.org. If a request comes in for a sub-domain of mydomain.org that doesn't have its own folder, we'll route it through that folder. You can combine this feature with the urlSiteContents feature, or script-implemented pages.
We also set the <ahref="http://stackoverflow.com/questions/19084340/real-life-usage-of-the-x-forwarded-host-header">X-Forwarded-Host</a> and <ahref="https://en.wikipedia.org/wiki/X-Forwarded-For">X-Forwarded-For</a> headers on urlSiteContents requests.
New prefs and config values that allow you to disable processing of scripts and Markdown files. By setting the values in prefs.json, you control all domains on the server. And by adding the values to config.json, in the folder the site is served from, you can turn them off selectively by site. I needed to turn off script processing for .js files served from <ahref="https://github.com/scripting/river4">River4</a>, to make it possible to serve a full river from PagePark.
With this release you can serve domains whose content is stored elsewhere on the web.
There's a new optional element of config.json, urlSiteContents. Its value is the URL of a directory on the web that stores the content of the domain.
As an experiment, I mapped the domain my.this.how to point to a folder in my public Dropbox folder. Here's how I did it.
1. First I created a new sub-folder of my public Dropbox folder called pageParkDemo. I put one file in that folder, index.html.
2. I used the web interface of my domain registrar to point my.this.how at the IP address of my PagePark server.
3. On my PagePark server, I created a sub-folder of the domains folder called my.this.how.
4. I created a <ahref="https://gist.github.com/scripting/f06e32acdee685510211">config.json file</a> in that folder, pointing to the pageParkDemo folder in my Dropbox folder.
Obviously it can get a lot more elaborate, and you can store the content anywhere, not just in a Dropbox folder. The key is that the content be accessible over the web.
A new feature for pages implemented as scripts. If the script returns the value <i>undefined</i> PagePark will not return a value to the HTTP client, it assumes that the script will do this.
To make it possible for a script page to return a value to the client, there's a new built-in function httpReturn. It takes two parameters, the value, a string, and the type, a MIME type. For example you might have a script that makes an HTTP request and returns a value based on the result to the caller.
Here's an <ahref="https://gist.github.com/scripting/a3a8232d193ea88e04ba">example script</a> that illustrates.
New feature: If there's a file called config.json in a domain folder, we read it on every request, and values in that file can change the behavior of the server. The first feature allows you to do a whole-site redirect. Useful if you want to have several names map to the same content. Here's an <ahref="https://gist.github.com/scripting/27be2d8be40577ad0fdf">example</a> of the config.json file that maps a domain to <ahref="http://nodestorage.io/">nodestorage.io</a>.
The default port the server boots up on is now 1339. Previously it was 80, which is the standard port for HTTP, but on many OSes this requires PagePark to be running in supervisor mode. I added docs above to explain this.
Changed package.json so that only *request* and *marked* were listed as dependencies. Apparently the others are included in Node without having to list them.
Instead of keeping our own MIME type table, we use the Node *mime* package, which is also included as a dependency in the package.json file.
Please post a note on the <ahref="https://groups.google.com/forum/#!forum/server-snacks">Server Snacks</a> mail list or post an issue <ahref="https://github.com/scripting/pagePark/issues">here</a>.
