bandolier-website/rendered/docs/quick-start.md

{ "author": "Zed A. Shaw", "date": "Dec 30, 2022", "has_image": false, "tag": "Demo", "icon": "baby", "summary": "How to get started with The Bandolier in about 20 minutes." }

Getting Started in About 20 Minutes

This quick guide will get the bandolier-template installed on your computer and ready to modify. I say it should take about 20 minutes, but in the world of JavaScript that depends on a lot of factors. If you have a slow network then installing the node_modules could take 60+ minutes. If you are on Windows you could run into weird errors with node-gyp. Oh wait, if you're on any platform that steaming pile of combustible garbage called node-gyp will probably cause problems.

"Twenty minutes" is a rough guess if nothing goes wrong. If something does go wrong, please let me know so I can sort out what's happening. These instructions are tested on macOS, Windows 10, ArchLinux and alpine Linux.

This is a very early release to have people try it and tell me how it worked on their platform. I test this OSX, Windows, and Linux but tell me if you run into installation issues, especially with regards to node-gyp.

Warning for Linux Die Hards

I don't support every flavor of Linux so if you're running a variant of Nix "OS" inside a Qubes "OS" variant that runs a modified DRM free version of Wayland with only "totally free" video protocols installed and a broken sound card because your Emacs "OS" has a bug that disables sockets inside Nix "OS" when Qubes "OS" can't move a window in Wayland, and that's why you can't run Chrome and use Brave instead then that's why I don't support every Linux.

It's alright to be weird and try new things, but don't expect everyone to support your weirdness like you're doing nothing unusual with your "DRM free FLOSS/GNU Linux Qubes OS Nix OS ARM 64 computer running Brave."

Pre-requisites

You will need the following software installed:

1. redis server. On Windows you can install Windows Subsystem for Linux, install Ubuntu, and then install redis inside there. Once you do that you can run redis right from inside PowerShell with wsl -u root /etc/init.d/redis-server start. Later you can use the node bando.js devsvc and it will run everything for you.
2. Node 18 LTS or Node 16 LTS. This should be available on every platform and easy to install, but check if you already have it installed.
3. Any text editor you like.
4. Any terminal program you like, but definitely PowerShell on windows.
5. As you install node modules you may find you need additional software for your platform. For example, some Linux systems need you to install various sqlite3 libraries.

Installation

First, install the ljsthw-bandolier module:

npm install git+https://git.learnjsthehardway.com/learn-javascript-the-hard-way/ljsthw-bandolier.git

This command installs an installer module directly from the project's git service. Doing this creates a command you'll use to manage installs and updates of the Bandolier Template. The code for the framework lives there.

You should now be able to do this:

npx bando-up --version
npx bando-up --help

If you can't then refer to the documentation for ljsthw-bandolier, especially if you get errors regarding SSL certificates on Windows.

If you can run the npx bando-up command then use it to create your first project:

npx bando-up create my-project

This will checkout the Bandolier Template git repository into the my-project directory. Once it's done you can move on to Configuration.

The bando-up command is also used during the Learn JS the Hard Way course to install exercise code, sample projects, and other tools. If you want to see how it works and create your own "off grid" npm setup then read How to Create Your Own npm init and Get Off npmjs.com.

Configuration

If you checked out your first project into my-project then do this:

cd my-project
npm install

That will move you into the project directory and install all of the required software. If you get errors with node-gyp see below on how to fix them (maybe).

Once npm install finishes you can configure the application:

npm run knex migrate:latest

This will setup your database, and then you can finally do the initialize command to finish the setup:

node bando.js init

This will configure some items, rerun the migrations just in case, and copy any template files that may have been missed. It will skip files you already have, and then wait for you to start the app in another window.

On some versions of Linux you may get a weird error Couldn't get a file descriptor referring to the console when the commands/init.js script tries to run Node's execSync. Currently I have no idea why this error happens, as it doesn't happen on any Linux I run. If you have an idea what's causing this, please let me know.

Open another Terminal window and start the app in DANGER_ADMIN mode:

npm run DANGER_ADMIN

This starts the app in developer mode, but it's called DANGER_ADMIN so you know for sure you should not be running it this way in production on the internet.

Once it's running, switch back to the other window and hit ENTER. This will open your browser to http://localhost:5001 so you can register a first user. Fill out the registration form with a fake user and hit the Register and Continue to Payment button.

When your developer user is first registered it's not an administrator. Switch back to the window running node bando.js init and hit ENTER again. The command will then set the first user to admin=1 and report FINISHED!.

The final test is switch back to the browser and refresh the page to see that you are now administrator.

Install Error with Playwright

In theory you shouldn't need playwright to download browsers, so just tell it not to.

PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1 npm install

This isn't as needed on Linux as OSX, because the Playwright project forces people to upgrade their OS by claiming all versions of their project can't support any slightly older Safari versions. You also just don't need Safari, so skip the download.

By default I use the Chrome browser since it's usually faster, cross platform, and downloads without any problems. If you want to use something else change the code in lib/testing.js:playstart.

Errors Related to node-gyp

If this doesn't work then this might https://github.com/nodejs/node-gyp/blob/main/docs/Updating-npm-bundled-node-gyp.md which says that npm updates don't update the node-gyp it uses internally. Depending on your version you will have to run different commands, but this works for 7,8, and 9 versions:

npm explore npm/node_modules/@npmcli/run-script -g -- npm_config_global=false npm install node-gyp@latest

There's also an issue with installing node-pre-gyp where it seems you need to install @mapbox/node-pre-gyp and also node-pre-gyp.

The bando.js Script

If you're on OSX or Linux you can just run ./bando.js directly rather than node bando.js. If you're on Windows you can use the PowerShell script ./bando instead. To make these instructions work for people who skim I'm using node bando.js.

This command is a simple "runner" script that looks in the directory commands for all .js files to provide them as commands. You can go in there right now and look at the commands/init.js file to see what it did. The code is fairly simple, and a good one to study first. Feel free to look at the other commands as well.

You can get help and list commands with --help or the help command:

node bando.js --help
node bando.js help

Then each command listed has its own --help option:

node bando.js init --help
Usage: bando init [options]

Initializes a fresh bando project

Options:
  --force      DANGER! overwite config files with templates
  --templates  what to use as a template directory
  -h, --help   display help for command

The code for each command is fairly small and educational, so take some time to look through them. You might like commands/convert.js to see how I process videos with ffmpeg, and commands/codedoc.js to see how I use acorn to parse code to generate the API Docs.

Quick Tour

The directory structure of The Bandolier is flat to avoid nested hierarchies. Nesting directories deeply makes it more difficult to work on the code, and more confusing for beginners. By flattening the structure out you have direct access to every component, and each component is clearly separated. For example, rather than try to make one directory of .svelte files do both dynamic and static content there is a directory client for the dynamic client "app", and rendered for the rendered pages. This keeps thing digestible and easy to study since each directory does mostly one thing.

Here's what each directory contains:

• admin - This is where the admin control panel lives.
• api - The JSON api handlers live here.
• bando.js - This is your main management script.
• bando.ps1 - This is a Windows compatible version of the script.
• build.json - A build configuration used by the commands/build.js command that runs esbuild.
• build.prod.json - The production build configuration used in the npm run build command.
• client - This is where the main web application lives, and is a dynamic Svelte front-end.
• commands - The bando.js script runs commands out of here.
• coverage - You won't see this at the start, but if you run the coverage commands then you'll see code coverage output here.
• debug - Various debugging outputs end up here.
• dev.sqlite3 - SQLite3 is the default database (PostgreSQL coming soon).
• dev.sqlite3-shm - You'll see this because the SQLite3 database is configured for performance.
• dev.sqlite3-wal - Same as above.
• emails - Your email templates and configurations are in here. Edit these to change how you email your users.
• knexfile.cjs - This is the database configuration using knex.js.
• lib - Various support utilities for are found in here, but are only for non-browser tools. Look in client/ for admin and client imports.
• media - If you do videos or audio then this can be a separate directory for media. You need this so you can wipe public/ at any time.
• migrations - The knex.js migrations. You'll see all that I've made over development so you have many examples.
• node_modules - Your modules when you run npm install.
• nodemon.json - The package.json uses nodemon to trigger builds when you change something. Esbuild handles this for admin and client code though.
• package-lock.json - npm makes this, and you can search in here to see exact versions of your packages and what depends on what.
• package.json - This configures the project. Take special node of the "type": "module" configuration, which configures node for ES6 style ESM imports.
• public - This directory should be safe to empty if you need to debug how things are being built. It is constructed from all of the other directories to create the content you would place on your webserver.
• queues - This directory has queue handlers, which use Bull to offload long running processing. Email sends, Discord bot handling, Paypal notifications, Stripe notifications, and Livestream notifications are handled by these.
• rendered - This is where the rendered static pages go, and you can look in rendered/pages/blog to see an example of using this.
• scripts - These are mostly scripts and "junk" other modules need, like Svelte's TypeScript configuration script, or PM2 configuration examples.
• secrets - NEVER PUT THIS IN GIT. This is where your secret configuration files go. This will contain Payment keys, Discord keys, and other special configurations you don't want people to get.
• socket - This contains the socket.io handlers that give easy asynchronous communication with the browser.
• static - These are static files that need to be copied over to public/. It's things like icons, images, browser JavaScript code, etc.
• tests - Contains the automated ava tests for the application. The majority of them use Playwright to run the browser like a user to confirm things are still working.

As you can see, this includes almost everything you'd need to learn as a full stack developer. The code included isn't 100% feature complete, but it does have the minimum features necessary to make it work. That leaves room for you to improve it and make it easier to study.

Built-in Development Tools

The Bandolier tries to use the web browser to make web development easier. When you first go to 127.0.0.1:5001/client/#/ you'll see a mostly blank page with a bunch of icons in the top header. Click on the keyboard. Now you're in the built-in administrator that provides:

• Simple database administration that understands the SQLite3 schema. It's good enough to manage your application for quite a while.
• Quick Email testing tools to make sure your DNS entries are probably correct.
• Quick email sending test tool to test your email configuration.
• Viewing anonymized web server statistics generated by the bando.js loganalyzer command.
• Full Icon browser/finder for the included lucide icon set. This is probably all of the icons you'll ever need, and the node bando.js icons command will package up your production set of .svg icons for extra speed.
• Component browser for all of the components in client/components, with demos, quick docs, and demo code.
• Browser for all of your api/ and socket/ routes so you can see what's configured and look at the code.
• The "Djenterator", which is a visual code template generator. It has a simple JSON file you edit to change the code and then you copy it out to get started. Or, use the node bando.js djent command to do it from the command line.

The ctrl-atl-b Hotbar

You can also type ctlr-alt-b and a little "hotbar" will pop-up with quick links to common things you need quickly like errors, icons, routes, etc. This makes development faster as you can access tools directly from the pages you're working on.

Making Your First Changes

We'll make some minor modifications to the framework to learn the basics. First, we'll just change the home page:

1. Open the client/pages/Home.svelte file in your editor.
2. Change the Welcome! text to say what you want.
3. Type ctrl-alt-b and click on the Icons tool.
4. Find an interesting icon, and click on its block. The code for that icon is now copied.
5. Go back to Home.svelte in your editor and paste that Icon somewhere.
6. Save and it should reload (hopefully), if not refresh.

The reload code is new and found in commands/build.js. It uses the reload functionality found in esbuild so it might not reliably reload on your platform. Please let me know if you have any ideas why it fails on your setup.

Running the Tests

The bando uses the excellent ava testing framework and the mediocre Playwright browser testing system. It also includes a way to get coverage from all layers of the framework during testing, but for the Quick Start we'll only run the tests:

npm run test

This should run the test and hide the browser window used for the tests. If you want to see the browser while the tests run then set the PLAYVIEW=1 environment variable:

PLAYVIEW=1 npm run test

If you're on Windows then you do this:

$env:PLAYVIEW=1
npm run test

This is different from the Unix/OSX command because it sets the environment variable for the whole session. You can unset it by setting it to an empty string:

$env:PLAYVIEW=''

Then the browser will stop showing. Read more about Windows environment variables here in the docs.

Fixing Errors

The bando comes with a rudimentary error reporting system. This works about 80% of the time due to how Svelte and esbuild reports errors and how browser refresh works:

1. You'll get error reports in the browser whenever you run with npm run DANGER_ADMIN.
2. The errors are actually pulled from Svelte/esbuild by the commands/build.js command and written to debug/admin_errors.json and debug/client_errors.json. These files also contain all of the warnings but they aren't shown in the Browser yet.
3. On a refresh the app will check for these files by hitting the /api/devtools/info/ handler. If this handler returns errors then you'll get a little pop-up in the bottom right and you can click on it to see the errors.
4. If this isn't working automatically then you can view the errors from the most recent build by typing ctrl-alt-b and clicking the Errors icon.

To test out the error reporting break the client/pages/Home.svelte and see what happens:

1. Open client/pages/Home.svelte in your editor.
2. Break this page by removing some important character.
3. It should refresh and display an error pop-up. If not hit ctrl-alt-b and click the Errors icon.
4. Click on the error pop-up and it will open the browser to an error view.
5. Fix the error, it should reload and the error should go away, but sometimes you have to refresh (bug).

Adding a New Client Page

The next thing is to learn how to add a new page. In the bando the concept of a "page" is separated into "client pages" and "rendered pages." A client page is your Single Page App (SPA) page, and is bundled into a single .js file using esbuild using commands/build.js. The rendered pages are your traditional "static" pages, but they work more like a Multi-Page App (MPA). The rendered pages (MPA) still use Svelte, have access to all of the components, but render to a static file.

In this part of the Quick Start we'll only add a simple page to the client pages. The Long Start start will go through every feature and cover the rendered pages.

1. Create an empty client page using: ./bando.js djent --template ./static/djenterator/client.svelte --output client/pages/Test.svelte.
2. There is now a file at client/pages/Test.svelte. Open it in your editor to have a look.
3. Open client/routes.js and copy other pages to add this new Test.svelte:
   1. Import it import Test from "./pages/Test.svelte";
   2. Add it to the routes listing at the end "/test/": Test, .
4. Go to http://127.0.0.1:5001/client/#/test/ and your page shows up.

You can also use the Djenterator to visually see all of the node bando.js djent templates. You can also use this to modify the variables that craft the file.

The client/routes.js file is a debatable choice. It might be possible to generate the routes from other sources--possibly as an option in the .svelte file--but there's so many web frameworks that have this kind of routing configuration that it might be educational to keep it. Let me know what you think.

Later Sessions

When you return to work on your project you'll first need to start up redis and other services with:

node bando.js devsvc

This should work on macOS, and Windows (with WSL+redis installed). On Linux you should run redis-server using systemctl or whatever process manager you use. If you want to run it with devsvc then do this:

node bando.js devsvc --os linux

Then you can start your project like normal:

npm run DANGER_ADMIN

Next Steps

This Quick Start is designed to only get you up and running and make sure the basic features are working.