/* This is the analog to `lib/api.js` and provides features that help work consistently with the JSON data returned by `commands/api.js` handlers in `api/`. This library is designed so you can start working on your UI without anything in the backend. That means you can create "mock" routes using the `mock()` function, design your data along with your UI in the same file, and then "move" that configuration to an `api/` handler right after. ### Getting Started Easiest way to get a client going is to use the `bando.js djent` command: ```shell node bando.js djent --template ./static/djenterator/client.svelte --output client/pages/Test.svelte ``` You can also go to the [Djenterator](http://127.0.0.1:5001/admin/#/djenterator/) with your browser to visually alter the output. I recommend this if you already know what you want your data mock to start with. ### Updating Routes Once you run this example command you'll have a file in `client/pages/Test.svelte`. Currently you manually add these the file `client/routes.js` like this: ```javascript import Test from "./pages/Test.svelte"; export default { "/test/": Test, } ``` If you open this file now you'll see many other routes that make up the stock web application. Obviously you'll "augment" this file, rather than use the above as the only contents. ### UI First After this you need to edit the `client/pages/Test.svelte` file, and use the sample data in the mock configuration: ```javascript api.mock({ "/api/user/profile": { "get": [200, {"message": "OK"}], } }); ``` You can then use the `client/api.js` functions to work with this pretend data to develop your UI. Just do your `api.get` (or `post`, or whatever) and augment the fake data you return as you create the visual experience. ### Craft Tests? Testing can come right at this point if you use the Playwright testing setup in `lib/testing.js`. Since you have the UI sort of working, you can take a bit of time writing a test and speed up the later steps. The idea is you write your test to click on all the buttons, fill out forms, and cause errors. When the tests mostly run with your fake UI then later development steps will be quicker because you'll automate testing that the new code works. For example, if you have a working test for a fake UI, then you craft a `api/` handler, your test will tell you that your new `api/` is properly replacing your fake data automatically. ### Migrate to `api/` When your UI is pretty good you can take the fake data you've been working on in `api.mock()` and simply move it to your `api/` handler. Just generate one using the [Djenterator](http://127.0.0.1:5001/admin/#/djenterator/) but plug in your fake data. This will craft a handler that returns this data, and in _theory_ this will then keep working with your UI. Once you have your fake data flowing out of the new `api/` handler then you can work on the form validation, logic, and other things you need to refine the UI further. ### Create Models The final step is to take your futher developed data and create a model for it in `lib/models.js`. This will usually require crafting a migration with: ```shell npm run knex migrate:migrate some_unique_description ```` ### Refine Tests If you made a UI test then in _theory_ most of your testing is done and just needs refinement. You should try to cause as many errors as you can, and then use the coverage system to make sure you're at _least_ running all of the code. It's hard to hit every line, but aim for as high as you can. After that, I recommend tests for your Models, but tests that hit the `api/` are usually low value as long as you're working the UI with Playwright. The reason is the _UI_ is already hitting the `api/` so other tests are largely duplicate. */ const MOCK_ROUTES = {}; import { user, cache_reset } from "./stores.js"; import { log } from "$/client/logging.js"; import Validator from 'Validator'; /* This performs a _client side_ validation of a form, updating the internal metadata so your `FormField.svelte` displays errors. You can look in `client/components/Login.svelte` for an example, but the data format is: ```javascript let form = { email: "", password: "", _valid: false, _errors: {}, _rules: { email: 'required|email', password: 'required' } } ``` One feature of `lib/api.js:API.validate()` is it returns the validation rules you create in the `api/` handler. This means you can keep validation rules in the most important place: the backend. You then submit your first form attempt, handle the error, and update this form with the form the backend returns. If you do this, then your UI pattern is: 1. No validation feedback on the first form fillout. This improve usability as it doesn't confuse the user with fake errors while they're typing. 2. Submit the form, then the `api/` handler validates, and returns the validation Rules in the response. 3. After this response your UI handles the validation with local fast UI feedback, and when it's correct submits it again to the `api/` handler. 4. Your `api/` handler then still keeps validating, but there's less useless network traffic and faster user feedback. I've found this pattern is better for security and usability. It's more usable because the user isn't slammed with useless validations while they type, giving them a chance to make edits and fix problems. It's more secure because the rules for validation and the true validation are all in the `api/` handler backend where it's required. The Validation rules can also come from the `lib/models.js:Model.validation` generator so they're based on the database schema. + `form Object` -- The form fields, with any validation settings. Things starting with `_` are considered internal. + `extra(form)` -- A callback you can use to do additional validation too complex for the rules. */ export const validate = (form, extra) => { if(form._rules) { let validation = Validator.make(form, form._rules); form._valid = validation.passes(); if(extra) extra(form); form._errors = validation.getErrors(); return form; } else { return form; } } /* Determines if the form can be submitted. If you read the docs for `validate` I say you should submit the first attempt, then validate the remaining ones in the browser (and backend). This function helps you do that as it will say `true` if there are no `form._rules` set. These only get set when you handle the response from the `api/` handler, so the first request will go through, then after that this function looks for `form._valid`. + `form Object` -- The form fields, with any validation settings. */ export const can_submit = (form) => { return form._rules === undefined || form._valid; } /* Cleans a form of junk before submitting to the `api/` handler. It removes + `form Object` -- The form fields, with any validation settings. + `extras Array` -- Additional fields to remove. */ export const clean_form = (form, extras=[]) => { form._errors = {}; // errors is accessed so needs to exist delete form._valid; delete form._rules; for(let field of extras) delete form[field]; } /* The default fetch_opts are `{ credentials: 'same-origin',}` to enforce authentication. Don't really change these here unless you know what you're doing. */ export const fetch_opts = { credentials: 'same-origin'}; /* Create a mock data response, so you can pretend you have a `api/` handler without having to actaully make one at first. This makes development easier since you don't need to bounce around between tons of files just to get a UI going. ### Config Format The primary key is the URL, and it contains an object mapping each `get`, `post`, etc. to the data returned. Here's an example: ```javascript api.mock({ "/api/user/profile": { "get": [200, {"message": "OK"}], } }); ``` You can have multiple URLs and multiple actions per URL, but the data returned is static. If you're at a point where you need to change the data response at random then it's time to write an `api/` handler. + `config Object` -- Configuration of routes to data response. */ export const mock = (config) => { for(let [route, value] of Object.entries(config)) { MOCK_ROUTES[route] = value; } } /* The internal version of `raw()` that does a mock call instead. The `raw()` functionw will call this if the URL is in `api.mock()`. Otherwise it's mostly internal. */ export const raw_mock = (url, raw_method, body, unauthed_action) => { let error; const method = raw_method.toLowerCase(); const config = MOCK_ROUTES[url]; const data = config[method]; if(data === undefined) { error = `Mock ${url}:${method} is not in your mocks list. Did you forget to add it to the api.mock call?`; } else if(data.length !== 2) { error = `Mock ${url}:${method} does not have the correct config. Must be [status (int), {data}], like [200, {message: 'OK'}].` } if(error) { log.error(error, MOCK_ROUTES); return [500, {message: error}]; } else { // this is the same as res.status below if(unauthed_action && (data[0] === 401 || data[0] === 403)) unauthed_action(); return data; } } /* JavaScript's URL parsing only things full complete URLs with ports and hosts are real URLs. This fakes it out. */ const parse_url_because_js_is_stupid = (url) => { try { // javascript is dumb as hell and thinks a typical /this/that is not a URL return new URL(url); } catch(error) { // so fake it for the mock by tacking on ... localhost then ignoring it return new URL(`http://localhost${url}`); } } /* All of the calls to `api/` handlers are mostly the same, so this one function does them all. This is then called by every request method like `get()` and `post()`. It's used internally so only access it if you're really desperate. The main thing to understand is that `client/api.js` tries to normalize as much as possible, including authentication. When the handler returns a 403/401 response this function will return the error like normal, but if you add the `unauthed_action()` callback then you can catch this and do a redirect to the login page. Look at the `client/components/LoggedIn.svelte` for an example of using this. ___FOOTGUN___: This calls `raw_mock` if the URL you requested is in a `api.mock()` specification. If you're having trouble with requests not going through delete your `api.mock()`. + `url string` -- URL to request. + `method string` -- ALL CAPS METHOD NAME LIKE POST GET. + `body Object` -- Body for the JSON request. + `unauthed_action()` -- Callback for what happens when a 403/401 is returned. */ export const raw = async (url, method, body, unauthed_action) => { const parsed = parse_url_because_js_is_stupid(url); // if there is a rules and valid is exactly false we should just run // the validation here and avoid the request if(body && body._rules && body._valid === false) { const res = validate(body); if(!can_submit(res)) { log.debug("Form invalid, won't HTTP submit."); return [400, res]; } else { clean_form(body); } } if(parsed.pathname in MOCK_ROUTES) { return raw_mock(parsed.pathname, method, body); } else { let options = { method, credentials: 'same-origin', headers: { 'Content-Type': 'application/json', }, } if(body) options.body = JSON.stringify(body); /* this handy little gem exists because fetch likes to go: * "the string did not match the expected pattern” * when the response body is not json...or at least that's * what I think it means. Now I get the text, and use it * for logging. */ let res = await fetch(url, options); let text = await res.text(); try { if(unauthed_action && (res.status === 401 || res.status === 403)) unauthed_action(); return [res.status, JSON.parse(text)]; } catch(error) { log.error(error, "Failed to parse reply body as JSON. Text is:", text, "error", error, "URL", url); return [500, {"message": "Exception processing request. See log.debug."}]; } } } /* Used for fetching basic resources, not JSON data from the API. Uses the same return signature as `raw()` but doesn't attempt a JSON conversion, and only does a "GET" request. It returns the result of `res.blob()` so you can convert it to a string or JSON as you need, or leave it if it's a binary file. + `url string` -- URL to get. + ___return___ `Array[status, Blob|{}]` -- Returns a Blob or when status is 500 a `{"message": "Error message"}`. */ export const blob = async (url) => { let data; try { let res = await fetch(url, fetch_opts); data = await res.blob(); return [res.status, data]; } catch(error) { log.error(error, "Failed to parse reply body as JSON. Text is:", data, "error", error, "URL", url); return [500, {"message": "Exception processing request. See log.debug."}]; } } /* The GET method request. To keep things consistent with the other requests this accepts a `data` parameter, but it URL encodes them and attaches them to the URL before calling `raw()`. + `url string` -- The url to request. + `data Object` -- Data to URL encode and append to the URL. + `unauthed_action()` -- Callback on authentication failure to let you redirect to a login. */ export const get = async (url, data, unauthed_action) => { const params = new URLSearchParams(data || {}); const param_url = `${url}?${params.toString()}`; return await raw(param_url, 'GET', undefined, unauthed_action); } /* The POST method request. Refer to `raw()` for more documentation. + `url string` -- The url to request. + `data Object` -- Data to URL encode and append to the URL. + `unauthed_action()` -- Callback on authentication failure to let you redirect to a login. */ export const post = async (url, data, unauthed_action) => { return await raw(url, 'POST', data, unauthed_action); } /* The PUT method request. Refer to `raw()` for more documentation. + `url string` -- The url to request. + `data Object` -- Data to URL encode and append to the URL. + `unauthed_action()` -- Callback on authentication failure to let you redirect to a login. */ export const put = async (url, data, unauthed_action) => { return await raw(url, 'PUT', data, unauthed_action); } /* The DELETE method request. Refer to `raw()` for more documentation. + `url string` -- The url to request. + `unauthed_action()` -- Callback on authentication failure to let you redirect to a login. */ export const del = async (url, unauthed_action) => { return await raw(url, 'DELETE', undefined, unauthed_action); } /* The OPTIONS method request. Refer to `raw()` for more documentation. + `url string` -- The url to request. + `unauthed_action()` -- Callback on authentication failure to let you redirect to a login. */ export const options = async (url, unauthed_action) => { return await raw(url, 'OPTIONS', undefined, unauthed_action); } /* Properly logs the user out of the backend _and_ frontend. The one major drawback to the SPA model is you have to sync the user's state with the backend. This cleans out the `client/stores.js:user` state, resets any caches being used, send a `get('/api/logout')`, and redirects the window to `/client/#/login`. */ export const logout_user = async () => { user.update(() => { return {authenticated: undefined} }); // really only a problem for multiple people // using the same browser, but clear it out cache_reset(); let [status, data] = await get('/api/logout'); if(status !== 200) log.error("Invalid status from logout", status, data); window.location.replace("/client/#/login"); } /* Gets the schema for the database, which isn't needed for most operations, but if you're working on anything that modifies the database or needs the database schema this is useful. ___WARNING___: This is only accessible to users with `admin=1` and blocked in the `api/admin/schema.js` handler. Don't think that because this is in `client/api.js` that the schema information should be exposed to the internet. The handler is read only, but you never know what weird things people can figure out from your schema. */ export const schema = async (table) => { let [status, tables] = await get('/api/admin/schema'); if(status == 200) { for(let t of tables) { if(t.name === table) { // this exits the function with the schema return t._columns; } } // this happens if the table isn't in the schema return undefined; } else { return undefined; } } export default { post, get, put, del, mock, options, fetch_opts, logout_user, validate, can_submit, clean_form, schema, blob }