# Introduction

The Bandolier (aka `bando`) is an educational web framework featured in the [Learn JavaScript the Hard Way](https://learnjsthehardway.com) course. The Bandolier contains all of the features a full stack developer would need to learn, but with smaller easier to understand implementations that are fully visible in the project. It includes implementations of:

* JSON APIs backends.
* Multi-page and Single-page web UIs.
* Queues for offloading processing.
* Video support with HLS, HTML, and WebTorrent Video.
* Built-in Database administration and a simple ORM.
* Authentication with passport.
* Websockets with socket.io.
* Pre-made CSS components and a simple CSS design method for non-designers.
* Unit testing and UI test automation.
* Automation commands to automate the boring tasks.
* Helpful video processing in JavaScript.
* Template generators to get started with code.
* Full but simple database administrator in the browser.
* Basic [Discord](https://discord.com) integration.
* Custom documentation generation for APIs using [acorn](https://github.com/acornjs/acorn).
* Includes common things you need like [lucide](lucide.dev) icons, Email DNS testing, web log analysis, in an easy to extend administrator tool.
* All implemented in reasonably sized pieces of code you can study on your own.

You can think of `bando` as a belt full of simple useful web development code and practices that help you learn how to get things done.  The idea is that everything in here is "similar" to features in other full stack web frameworks, but simplified so you can study them.  Once you learn what's here you should be able to learn most other features in other frameworks.

## About `bandolier-template`

This is the template project that's checked out and configured when you run the `npx bando-up` command from [ljsthw-bandolier](https://git.learnjsthehardway.com/learn-javascript-the-hard-way/ljsthw-bandolier#ljsthw-bandolier).  This is where the code really lives.

## Installation

First, install the [ljsthw-bandolier](https://git.learnjsthehardway.com/learn-javascript-the-hard-way/ljsthw-bandolier#ljsthw-bandolier):

```shell
npm install git+https://git.learnjsthehardway.com/learn-javascript-the-hard-way/ljsthw-bandolier.git
```

That will create a command you'll use to manage installs and updates of the [Bandolier Template](https://git.learnjsthehardway.com/learn-javascript-the-hard-way/bandolier-template).  The code for the framework lives there.

You should now be able to do this:

```shell
npx bando-up --version
npx bando-up --help
```

If you can't then refer to the documentation for `ljsthw-bandolier`, _especially if you get errors regarding SSL certificates on Windows_.

If you can run the `npx bando-up` command then use it to create your first project:

```shell
npx bando-up create my-project
```

This will checkout the [Bandolier Template](https://git.learnjsthehardway.com/learn-javascript-the-hard-way/bandolier-template) git repository into the `my-project` directory.  Once it's done you can move on to _Configuration_.

## Configuration

If you checked out your first project into `my-project` then do this:

```shell
cd my-project
npm install
```

That will move you into the project directory and install all of the required software.  If you get errors with `node-gyp` see below on how to fix them (maybe).

Once `npm install` finishes you can configure the application:

```shell
npm run knex migrate:latest
```

This will setup your database, and then you can finally do the initialize command to finish the setup:

```shell
node bando.js init
```

This will configure some items, rerun the migrations just in case, and copy any template files that may have been missed.  It will skip files you already have, and then wait for you to start the app in another window.

Open another Terminal window and start the app in `DANGER_ADMIN` mode:

```shell
npm run DANGER_ADMIN
```

This starts the app in developer mode, but it's called `DANGER_ADMIN` so you know for sure you should _not_ be running it this way in production on the internet.

Once it's running, switch back to the other window and hit `ENTER`.  This will open your browser to `http://localhost:5001` so you can register a first user.  Fill out the registration form with a fake user and hit the `Register and Continue to Payment` button.

When your developer user is first registered it's not an administrator. Switch back to the window running `node bando.js init` and hit `ENTER` again.  The command will then set the first user to `admin=1` and report `FINISHED!`.

The final test is switch back to the browser and refresh the page to see that you are now administrator.

## Install Error with Playwright

In theory you shouldn't need playwright to download browsers, so just tell it not to.

```shell
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1 npm install
```

This isn't as needed on Linux as OSX, because the Playwright project forces people to upgrade their OS by claiming all versions of their project can't support any slightly older Safari versions.  You also just don't need Safari, so skip the download.

## Errors Related to `node-gyp`

If this doesn't work then this might https://github.com/nodejs/node-gyp/blob/main/docs/Updating-npm-bundled-node-gyp.md which says that npm updates don't update the node-gyp it uses internally. Depending on your version you will have to run different commands, but this works for 7,8, and 9 versions:

```shell
npm explore npm/node_modules/@npmcli/run-script -g -- npm_config_global=false npm install node-gyp@latest
```

There's also an issue with installing node-pre-gyp where it seems you need to install @mapbox/node-pre-gyp and also node-pre-gyp.

## The `bando.js` Script

If you're on OSX or Linux you can just run `./bando.js` directly rather than `node bando.js`.  If you're on Windows you can use the PowerShell script `./bando` instead.  To make these instructions work for people who skim I'm using `node bando.js`.

This command is a simple "runner" script that looks in the directory `commands` for all `.js` files to provide them as commands.  You can go in there right now and look at the `commands/init.js` file to see what it did. The code is fairly simple, and a good one to study first.  Feel free to look at the other commands as well.

## Quick Tour

_NOTE_: This is a very early release to have people try it and tell me how it worked on their platform. I test this OSX, Windows, and Linux but tell me if you run into installation issues, especially with regards to `node-gyp`.

The directory structure of The Bandolier is flat to avoid nested hierarchies.  Nesting directories deeply makes it more difficult to work on the code, and more confusing for beginners.  By flattening the structure out you have direct access to every component, and each component is clearly separated.  For example, rather than try to make one directory of `.svelte` files do both dynamic and static content there is a directory `client` for the dynamic client "app", and `rendered` for the rendered pages.  This keeps thing digestible and easy to study since each directory does mostly one thing.

Here's what each directory contains:

* `admin` - This is where the admin control panel lives.
* `api` - The JSON api handlers live here.
* `bando.js` - This is your main management script.
* `bando.ps1` - This is a Windows compatible version of the script.
* `build.json` - A build configuration used by the `commands/build.js` command that runs esbuild.
* `build.prod.json` - The production build configuration used in the `npm run build` command.
* `client` - This is where the main web application lives, and is a dynamic Svelte front-end.
* `commands` - The `bando.js` script runs commands out of here.
* `coverage` - You won't see this at the start, but if you run the coverage commands then you'll see code coverage output here.
* `debug` - Various debugging outputs end up here.
* `dev.sqlite3` - SQLite3 is the default database (PostgreSQL coming soon).
* `dev.sqlite3-shm` - You'll see this because the SQLite3 database is configured for performance.
* `dev.sqlite3-wal` - Same as above.
* `emails` - Your email templates and configurations are in here.  Edit these to change how you email your users.
* `knexfile.cjs` - This is the database configuration using [knex.js](https://knexjs.org/guide/).
* `lib` - Various support utilities for are found in here, but are only for _non-browser_ tools.  Look in `client/` for admin and client imports.
* `media` - If you do videos or audio then this can be a separate directory for media. You need this so you can wipe `public/` at any time.
* `migrations` - The `knex.js` migrations.  You'll see all that I've made over development so you have many examples.
* `node_modules` - Your modules when you run `npm install`.
* `nodemon.json` - The `package.json` uses [nodemon](https://nodemon.io) to trigger builds when you change something.  [Esbuild](https://esbuild.github.io) handles this for `admin` and `client` code though.
* `package-lock.json` - `npm` makes this, and you can search in here to see exact versions of your packages and what depends on what.
* `package.json` - This configures the project. Take special node of the `"type": "module"` configuration, which configures node for ES6 style ESM imports.
* `public` - This directory should be safe to empty if you need to debug how things are being built.  It is constructed from all of the other directories to create the content you would place on your webserver.
* `queues` - This directory has queue handlers, which use [Bull](https://github.com/OptimalBits/bull) to offload long running processing.  Email sends, Discord bot handling, Paypal notifications, Stripe notifications, and Livestream notifications are handled by these.
* `rendered` - This is where the rendered static pages go, and you can look in `rendered/pages/blog` to see an example of using this.
* `scripts` - These are mostly scripts and "junk" other modules need, like Svelte's TypeScript configuration script, or PM2 configuration examples.
* `secrets` - _NEVER PUT THIS IN GIT_. This is where your secret configuration files go. This will contain Payment keys, Discord keys, and other special configurations you don't want people to get.
* `socket` - This contains the [socket.io](https://socket.io) handlers that give easy asynchronous communication with the browser.
* `static` - These are static files that need to be copied over to `public/`. It's things like icons, images, browser JavaScript code, etc.
* `tests` - Contains the automated [ava](https://github.com/avajs/ava) tests for the application. The majority of them use [Playwright](https://playwright.dev) to run the browser like a user to confirm things are still working.

As you can see, this includes almost everything you'd need to learn as a full stack developer.  The code included isn't 100% feature complete, but it does have the minimum features necessary to make it work.  That leaves room for you to improve it and make it easier to study.

## You Can Change or Remove Almost Everything

The Bandolier is designed for you to learn new technologies without having to create an entire web application from scratch.  If you want to try [Actix](https://actix.rs) as a backend JSON server then just make it return the same JSON on the same URLs `api/` and it should work.  Don't like the ORM? You could probably replace it fairly easily.  Want to learn how to use React?  Just start your own React app in a new directory, and slowly bring over what you need from `client/` but in React form. Don't like my CSS?  All of the HTML is very simple and semantic, so you can craft almost any classless CSS you want.  Converting to something like [bootstrap](https://getbootstrap.com) would be some work, but you can use the `client/components` as a list of things to replicate in your version.  Probably the most difficult thing to replace is the is the `client/` directory, since `admin/` and `rendered/` use components, but if you really want to learn a front-end framework then replicating all of the components in `client/components` would be an incredible way to learn.

The reason this works is because the code in `api/` doesn't really depend on anything in `client/` or `admin/`.  If it uses something in `queue/` it's through a library so you can replace it easily, or possibly even just use the [Bull](https://github.com/OptimalBits/bull) from your favorite backend language.  Bull just uses redis, and you could just replace all of that.  There's also a large test suite in `tests/` that you can keep running to make sure your changes are working.

Finally, all of the code is reasonably sized.  Most files are about 100 - 500 lines of code with not too much convolution or deep references.  You could probably pick a random file and spend a few hours to study it.  Many of the source files also teach a different technique.  For example, the `client/fsm.js` file implements a simple Finite State Machine, which is used to make HLS video work in `client/components/HLSVideo.svelte`.  Then in `client/components/WTVideo.svelte` there's more traditional callback style for handling WebTorrent videos so you can learn both techniques.

Since all of this code is right there, and there's lots of tests, you can feel free to experiment while you study it. When you want to use it for something real, you can use what's there, and improve what you need for your application.

## Built-in Development Tools

The Bandolier tries to use the web browser to make web development easier.  When you first go to [127.0.0.1:5001](127.0.0.1:5001) you'll see a mostly blank page with a bunch of icon in the top header.  Click on the keyboard.  Now you're in the built-in administrator that provides:

* Simple database administration that understands the SQLite3 schema.  It's good enough to manage your application for quite a while.
* Quick Email testing tools to make sure your DNS entries are probably correct.
* Quick email sending test tool to test your email configuration.
* Viewing anonymized web server statistics generated by the `bando.js loganalyzer` command.
* Full Icon browser/finder for the included [lucide](https://lucide.dev) icon set. This is probably all of the icons you'll ever need, and there's code you can use to package up your own set of `.svg` icons if you want.
* Component browser for all of the components in `client/components`, with demos, quick docs, and demo code.
* Browser for all of your `api/` and `socket/` routes so you can see what's configured and look at the code.
* The "Djenterator", which is a visual code template generation.  It has a simple JSON file you edit to change the code and then you copy it out to get started.

You can also type `ctlr-alt-b` and a little "hotbar" will pop-up with quick links to common things you need quickly like errors, icons, routes, etc. This makes development faster as you can access tools directly from the pages you're working on.

## Making Your First Changes

The documentation is light right now, but try this:

1. Open the `client/pages/Home.svelte` file in your editor.
2. Change the `Welcome!` text to say what you want.
3. Type ctrl-alt-b and click on the Icons tool.
4. Find an interesting icon, and click on its block. The code for that icon is now copied.
5. Go back to `Home.svelte` and paste that Icon somewhere.
6. Save and it should reload (hopefully), if not refresh.
7. Break this page by removing some important character.
8. It should refresh and display an error pop-up.
9. Click on the error pop-up and it will open the browser to an error view.
10. Fix the error, it should reload and the error should go away, but sometimes you have to refresh (bug).
11. Look at `client/routes.js` to see where pages are configured for each URL.
12. `ctrl-alt-b` and go to Library. This is the library of component demos for everything in `client/components`.  Browse around and try them out in the `client/pages/Home.svelte`.
13. Look at the code in `api/register.js` to see a good example of a JSON API.
14. Look at `lib/ormish.js` for the simple ORM(ish) built on top of [knexjs](knexjs.org). There's some fancy tricks in there too.
14. Look at `lib/models.js` for examples of using ORMish.
15. Click on the "keyboard" icon in the header (must be admin) to view other admin features, especially the `Tables` admin for the database.
16. Then continue studying anything you find interesting and trying to change the existing pages.

This should give you a quick crash course in how the framework operates while I work on better documentation and instructions.

## Current Status

This is a first rough ALPHA release to get the project published and get some victi`^H^H^H^H^H`...users testing it and telling me how it works so far.  You aren't expected to be able to figure it out since I haven't completely documented it, but you should feel free to explore the code, study what's there, and see what you can do.  The next release will have reference documentation and more code cleaning, then after that videos demonstrating the important features and showing how to use the tools included.

After that, further development will be focused on supporting the [Learn JavaScript the Hard Way](https://learnjsthehardway.com) course where students will use `bando` to create their own projects, and also learn how to implement the important elements on this stack.  Once the course is done I'll start writing more public documentation for the framework as well as more demonstrations of using it to implement common websites.