bandolier-website/rendered/docs/quick-start.md

{
  "author": "Zed A. Shaw",
  "date": "Dec 30, 2022",
  "has_image": false,
  "tag": "Demo",
  "icon": "baby",
  "summary": "How to get started with The Bandolier in about 20 minutes."
}
------
# Getting Started in About 20 Minutes

This quick guide will get the [bandolier-template](https://git.learnjsthehardway.com/learn-javascript-the-hard-way/bandolier-template) installed on your computer and ready to modify.
I say it should take about 20 minutes, but in the world of JavaScript that depends on a lot of factors.
If you have a slow network then installing the `node_modules` could take 60+ minutes.  If you are on
Windows you could run into weird errors with `node-gyp`.  Oh wait, if you're on _any_ platform that steaming pile of combustible garbage called `node-gyp` will probably cause problems.

"Twenty minutes" is a rough guess if nothing goes wrong.  If something does go wrong, please let me know so I can sort out what's happening.  These instructions are tested on macOS, Windows 10, ArchLinux and alpine Linux.

<div>
<callout class="warning">
<p>This is a very early release to have people try it and tell me how it worked on their platform. I test this OSX, Windows, and Linux but tell me if you run into installation issues, especially with regards to <code>node-gyp</code>.</p>

<p><b>M1/M2 MAC USERS BEWARE!</b> Your laptop has big problems with many open source projects.  If you experience problems with libraries, install, or similar then please tell me.  I don't know if I can fix them, but I'll do what I can.  Then go find another computer to use.
</p>
</callout>
</div>

## Warning for Linux Die Hards

I don't support every flavor of Linux so if you're running a variant of Nix "OS" inside a Qubes "OS" variant that runs a modified DRM free version of Wayland with only "totally free" video protocols installed and a broken sound card because your Emacs "OS" has a bug that disables sockets inside Nix "OS" when Qubes "OS" can't move a window in Wayland, and that's why you can't run Chrome and use Brave instead then that's why I don't support every Linux.

It's alright to be weird and try new things, but don't expect everyone to support your weirdness like you're doing nothing unusual with your "DRM free FLOSS/GNU Linux Qubes OS Nix OS ARM 64 computer running Brave."

## Pre-requisites

You will need the following software installed:

1. [redis server](https://redis.io).  On Windows you can install [Windows Subsystem for Linux](https://learn.microsoft.com/en-us/windows/wsl/install), install Ubuntu, and then install `redis` inside there.  Once you do that you can run `redis` right from inside `PowerShell` with `wsl -u root /etc/init.d/redis-server start`.  Later you can use the `node bando.js devsvc` and it will run everything for you.
2. [Node 18 LTS or Node 16 LTS](https://nodejs.org/en/blog/release/).  This should be available on every platform and easy to install, but check if you already have it installed.
3. Any text editor you like.
4. Any terminal program you like, but _definitely_ `PowerShell` on windows.
5. As you install `node modules` you may find you need additional software for your platform.  For example, some Linux systems need you to install various `sqlite3` libraries.

## Installation

First, install the [ljsthw-bandolier](https://git.learnjsthehardway.com/learn-javascript-the-hard-way/ljsthw-bandolier#ljsthw-bandolier) module:

```shell
npm install -g git+https://git.learnjsthehardway.com/learn-javascript-the-hard-way/ljsthw-bandolier.git
```

This command installs an installer module directly from the project's git service.  Doing this creates a command you'll use to manage installs and updates of the [Bandolier Template](https://git.learnjsthehardway.com/learn-javascript-the-hard-way/bandolier-template).  The code for the framework lives there.

You should now be able to do this:

```shell
npx bando-up --version
npx bando-up --help
```

If you can't then refer to the documentation for `ljsthw-bandolier`, _especially if you get errors regarding SSL certificates on Windows_.

If you can run the `npx bando-up` command then use it to create your first project:

```shell
# see Next Steps at the end regargind --keep-git here
npx bando-up create --keep-git my-project
```

This will checkout the [Bandolier Template](https://git.learnjsthehardway.com/learn-javascript-the-hard-way/bandolier-template) git repository into the `my-project` directory.  Once it's done you can move on to _Configuration_.

<div><callout class="info"><span>
The <code>bando-up</code> command is also used during the <a href="https://learnjsthehardway.com/">Learn JS the Hard Way</a> course to install exercise code, sample projects, and other tools.  If you want to see how it works and create your own "off grid" <code>npm</code> setup then read <a href="https://learnjsthehardway.com/blog/19-how-to-create-your-own-npm-init-and-get-off-npmjs/">How to Create Your Own <code>npm init</code> and Get Off npmjs.com</a>.
</span></callout></div>

## Configuration

If you checked out your first project into `my-project` then do this:

```shell
cd my-project
npm install
```

That will move you into the project directory and install all of the required software.  If you get errors with `node-gyp` see below on how to fix them (maybe).

Once `npm install` finishes you can configure the application:

```shell
npm run knex migrate:latest
```

This will setup your database, and then you can finally do the initialize command to finish the setup:

```shell
node bando.js init
```

This will configure some items, rerun the migrations just in case, and copy any template files that may have been missed.  It will skip files you already have, and then wait for you to start the app in another window.

<div>
<callount class="warning"><span>
On some versions of Linux you may get a weird error <code>Couldn't get a file descriptor referring to the console</code> when the <code>commands/init.js</code> script tries to run Node's <code>execSync</code>.  Currently I have no idea why this error happens, as it doesn't happen on any Linux I run.  If you have an idea what's causing this, please let me know.
</span></callout>
</div>

Open another Terminal window and start the app in `DANGER_ADMIN` mode:

```shell
npm run DANGER_ADMIN
```

This starts the app in developer mode, but it's called `DANGER_ADMIN` so you know for sure you should _not_ be running it this way in production on the internet.

Once it's running, switch back to the other window and hit `ENTER`.  This will open your browser to `http://localhost:5001` so you can register a first user.  Fill out the registration form with a fake user and hit the `Register and Continue to Payment` button.

When your developer user is first registered it's not an administrator. Switch back to the window running `node bando.js init` and hit `ENTER` again.  The command will then set the first user to `admin=1` and report `FINISHED!`.

The final test is switch back to the browser and refresh the page to see that you are now administrator.

### Install Error with Playwright

In theory you shouldn't need playwright to download browsers, so just tell it not to.

```shell
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1 npm install
```

This isn't as needed on Linux as OSX, because the Playwright project forces people to upgrade their OS by claiming all versions of their project can't support any slightly older Safari versions.  You also just don't need Safari, so skip the download.

<div><callout class="warning"><span>
By default I use the Chrome browser since it's usually faster, cross platform, and downloads without any problems.  If you want to use something else change the code in <code>lib/testing.js:playstart</code>.
</span></callout></div>

### Errors Related to `node-gyp`

If this doesn't work then this might https://github.com/nodejs/node-gyp/blob/main/docs/Updating-npm-bundled-node-gyp.md which says that `npm` updates don't update the `node-gyp` it uses internally. Depending on your version you will have to run different commands, but this works for 7,8, and 9 versions:

```shell
npm explore npm/node_modules/@npmcli/run-script -g -- npm_config_global=false npm install node-gyp@latest
```

There's also an issue with installing `node-pre-gyp` where it seems you need to install `@mapbox/node-pre-gyp` and also `node-pre-gyp`.

## The `bando.js` Script

If you're on OSX or Linux you can just run `./bando.js` directly rather than `node bando.js`.  If you're on Windows you can use the PowerShell script `./bando` instead.  To make these instructions work for people who skim I'm using `node bando.js`.

This command is a simple "runner" script that looks in the directory `commands` for all `.js` files to provide them as commands.  You can go in there right now and look at the `commands/init.js` file to see what it did. The code is fairly simple, and a good one to study first.  Feel free to look at the other commands as well.

You can get help and list commands with `--help` or the `help` command:

```shell
node bando.js --help
node bando.js help
```

Then each command listed has its own `--help` option:

```shell
node bando.js init --help
Usage: bando init [options]

Initializes a fresh bando project

Options:
  --force      DANGER! overwite config files with templates
  --templates  what to use as a template directory
  -h, --help   display help for command
```

The code for each command is fairly small and educational, so take some time to look through them.  You might like `commands/convert.js` to see how I process videos with `ffmpeg`, and `commands/codedoc.js` to see how I use [acorn](https://github.com/acornjs/acorn) to parse code to generate the [API Docs](/admin/#/docs/).

## Quick Tour

The directory structure of The Bandolier is flat to avoid nested hierarchies.  Nesting directories deeply makes it more difficult to work on the code, and more confusing for beginners.  By flattening the structure out you have direct access to every component, and each component is clearly separated.  For example, rather than try to make one directory of `.svelte` files do both dynamic and static content there is a directory `client` for the dynamic client "app", and `rendered` for the rendered pages.  This keeps thing digestible and easy to study since each directory does mostly one thing.

Here's what each directory contains:

* `admin` - This is where the admin control panel lives.
* `api` - The JSON api handlers live here.
* `bando.js` - This is your main management script.
* `bando.ps1` - This is a Windows compatible version of the script.
* `build.json` - A build configuration used by the `commands/build.js` command that runs esbuild.
* `build.prod.json` - The production build configuration used in the `npm run build` command.
* `client` - This is where the main web application lives, and is a dynamic Svelte front-end.
* `commands` - The `bando.js` script runs commands out of here.
* `coverage` - You won't see this at the start, but if you run the coverage commands then you'll see code coverage output here.
* `debug` - Various debugging outputs end up here.
* `dev.sqlite3` - SQLite3 is the default database (PostgreSQL coming soon).
* `dev.sqlite3-shm` - You'll see this because the SQLite3 database is configured for performance.
* `dev.sqlite3-wal` - Same as above.
* `emails` - Your email templates and configurations are in here.  Edit these to change how you email your users.
* `knexfile.cjs` - This is the database configuration using [knex.js](https://knexjs.org/guide/).
* `lib` - Various support utilities for are found in here, but are only for _non-browser_ tools.  Look in `client/` for admin and client imports.
* `media` - If you do videos or audio then this can be a separate directory for media. You need this so you can wipe `public/` at any time.
* `migrations` - The `knex.js` migrations.  You'll see all that I've made over development so you have many examples.
* `node_modules` - Your modules when you run `npm install`.
* `nodemon.json` - The `package.json` uses [nodemon](https://nodemon.io) to trigger builds when you change something.  [Esbuild](https://esbuild.github.io) handles this for `admin` and `client` code though.
* `package-lock.json` - `npm` makes this, and you can search in here to see exact versions of your packages and what depends on what.
* `package.json` - This configures the project. Take special node of the `"type": "module"` configuration, which configures node for ES6 style ESM imports.
* `public` - This directory should be safe to empty if you need to debug how things are being built.  It is constructed from all of the other directories to create the content you would place on your webserver.
* `queues` - This directory has queue handlers, which use [Bull](https://github.com/OptimalBits/bull) to offload long running processing.  Email sends, Discord bot handling, Paypal notifications, Stripe notifications, and Livestream notifications are handled by these.
* `rendered` - This is where the rendered static pages go, and you can look in `rendered/pages/blog` to see an example of using this.
* `scripts` - These are mostly scripts and "junk" other modules need, like Svelte's TypeScript configuration script, or PM2 configuration examples.
* `secrets` - _NEVER PUT THIS IN GIT_. This is where your secret configuration files go. This will contain Payment keys, Discord keys, and other special configurations you don't want people to get.
* `socket` - This contains the [socket.io](https://socket.io) handlers that give easy asynchronous communication with the browser.
* `static` - These are static files that need to be copied over to `public/`. It's things like icons, images, browser JavaScript code, etc.
* `tests` - Contains the automated [ava](https://github.com/avajs/ava) tests for the application. The majority of them use [Playwright](https://playwright.dev) to run the browser like a user to confirm things are still working.

As you can see, this includes almost everything you'd need to learn as a full stack developer.  The code included isn't 100% feature complete, but it does have the minimum features necessary to make it work.  That leaves room for you to improve it and make it easier to study.

## Built-in Development Tools

The Bandolier tries to use the web browser to make web development easier.  When you first go to [127.0.0.1:5001/client/#/](127.0.0.1:5001/client/#/) you'll see a mostly blank page with a bunch of icons in the top header.  Click on the keyboard.  Now you're in the built-in administrator that provides:

* Simple database administration that understands the SQLite3 schema.  It's good enough to manage your application for quite a while.
* Quick Email testing tools to make sure your DNS entries are probably correct.
* Quick email sending test tool to test your email configuration.
* Viewing anonymized web server statistics generated by the `bando.js loganalyzer` command.
* Full Icon browser/finder for the included [lucide](https://lucide.dev) icon set. This is probably all of the icons you'll ever need, and the `node bando.js icons` command will package up your production set of `.svg` icons for extra speed.
* Component browser for all of the components in `client/components`, with demos, quick docs, and demo code.
* Browser for all of your `api/` and `socket/` routes so you can see what's configured and look at the code.
* The "Djenterator", which is a visual code template generator.  It has a simple JSON file you edit to change the code and then you copy it out to get started. Or, use the `node bando.js djent` command to do it from the command line.

### The `ctrl-atl-b` Hotbar

You can also type `ctlr-alt-b` and a little "hotbar" will pop-up with quick links to common things you need quickly like errors, icons, routes, etc. This makes development faster as you can access tools directly from the pages you're working on.

## Making Your First Changes

We'll make some minor modifications to the framework to learn the basics.  First, we'll just change the home page:

1. Open the `client/pages/Home.svelte` file in your editor.
2. Change the `Welcome!` text to say what you want.
3. Type `ctrl-alt-b` and click on the Icons tool.
4. Find an interesting icon, and click on its block. The code for that icon is now copied.
5. Go back to `Home.svelte` in your editor and paste that Icon somewhere.
6. Save and it should reload (hopefully), if not refresh.

<div>
<callout><span>
The reload code is new and found in <code>commands/build.js</code>. It uses the reload functionality found in <a href="https://esbuild.github.io/">esbuild</a> so it might not reliably reload on your platform.  Please let me know if you have any ideas why it fails on your setup.
</span></callout>
</div>

## Running the Tests

The `bando` uses the excellent [ava](https://github.com/avajs/ava) testing framework and the mediocre [Playwright](https://playwright.dev/) browser testing system.  It also includes a way to get coverage from all layers of the framework during testing, but for the Quick Start we'll only run the tests:

```shell
npm run test
```

This should run the test and hide the browser window used for the tests.  If you want to see the browser while the tests run then set the `PLAYVIEW=1` environment variable:

```shell
PLAYVIEW=1 npm run test
```

If you're on Windows then you do this:

```shell
$env:PLAYVIEW=1
npm run test
```

This is different from the Unix/OSX command because it sets the environment variable for the whole session.  You can unset it by setting it to an empty string:

```shell
$env:PLAYVIEW=''
```

Then the browser will stop showing.  Read more about Windows environment variables here in [the docs](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_environment_variables?view=powershell-7.3).

## Fixing Errors

The `bando` comes with a rudimentary error reporting system.  This works about 80% of the time due to how Svelte and `esbuild` reports errors and how browser refresh works:

1. You'll get error reports in the browser whenever you run with `npm run DANGER_ADMIN`.
2. The errors are actually pulled from Svelte/esbuild by the `commands/build.js` command and written to `debug/admin_errors.json` and `debug/client_errors.json`. These files also contain all of the warnings but they aren't shown in the Browser yet.
3. On a refresh the app will check for these files by hitting the `/api/devtools/info/` handler.  If this handler returns errors then you'll get a little pop-up in the bottom right and you can click on it to see the errors.
4. If this isn't working automatically then you can view the errors from the most recent build by typing `ctrl-alt-b` and clicking the [Errors](http://127.0.0.1:5001/admin/#/errors/) icon.

To test out the error reporting break the `client/pages/Home.svelte` and see what happens:

1. Open `client/pages/Home.svelte` in your editor.
2. Break this page by removing some important character.
3. It should refresh and display an error pop-up. If not hit `ctrl-alt-b` and click the [Errors](http://127.0.0.1:5001/admin/#/errors/) icon.
4. Click on the error pop-up and it will open the browser to an error view.
5. Fix the error, it should reload and the error should go away, but sometimes you have to refresh (bug).

## Adding a New Client Page

The next thing is to learn how to add a new page.   In the `bando` the concept of a "page" is separated into "client pages" and "rendered pages."  A client page is your Single Page App (SPA) page, and is bundled into a single `.js` file using `esbuild` using `commands/build.js`.  The rendered pages are your traditional "static" pages, but they work more like a Multi-Page App (MPA).  The rendered pages (MPA) still use Svelte, have access to all of the components, but render to a static file.

In this part of the Quick Start we'll only add a simple page to the client pages.  The [Long Start](/client/#/long/) start will go through every feature and cover the rendered pages.

1. Create an empty client page using:  `./bando.js djent --template ./static/djenterator/client.svelte --output client/pages/Test.svelte`.
2. There is now a file at `client/pages/Test.svelte`. Open it in your editor to have a look.
3. Open `client/routes.js` and copy other pages to add this new `Test.svelte`:
    1. Import it `import Test from "./pages/Test.svelte";`
    2. Add it to the routes listing at the end `"/test/": Test,` .
4. Go to [http://127.0.0.1:5001/client/#/test/](http://127.0.0.1:5001/client/#/test/) and your page shows up.

You can also use the [Djenterator](http://127.0.0.1:5001/admin/#/djenterator/) to visually see all of the `node bando.js djent` templates.  You can also use this to modify the variables that craft the file.

<div>
<callout class="info"><span>
The <code>client/routes.js</code> file is a debatable choice.  It <b>might</b> be possible to generate the routes from other sources--possibly as an option in the <code>.svelte</code> file--but there's <b>so many</b> web frameworks that have this kind of routing configuration that it might be educational to keep it.  Let me know what you think.
</span></callout>
</div>

## Later Sessions

When you return to work on your project you'll first need to start up `redis` and other services with:

```shell
node bando.js devsvc
```

This should work on macOS, and Windows (with WSL+redis installed). On Linux you should run `redis-server` using `systemctl` or whatever process manager you use.  If you want to run it with `devsvc` then do this:

```shell
node bando.js devsvc --os linux
```

Then you can start your project like normal:

```shell
npm run DANGER_ADMIN
```

## Next Steps

One final thing you'll want to do is either delete the `.git` directory and make your own git like this:

```shell
# make sure you are in the my-project directory
pwd
# then remove .git and make your own
rm -rf .git
git init .
```

I suggest you also checkout the full `bandolier-template` for tracking updates like this:

```shell
# get out of your my-project directory if cd .. doesn't
cd ..
npx bando-up create --keep-git bandolier-template
```

If you create other projects later then drop the `--keep-git` so you don't have to do this step.

This Quick Start is designed to only get you up and running and make sure the basic features are working.