Skip to content

js

node packages updating

tl;dr

  1. > npm install depcheck -g - install depcheck globally
  2. > depcheck - check for redundant packages
  3. > npm un this-redundant-package - uninstall redundant packages (repeat for all redundant packages)
  4. Create a pull-request remove-redundant-packages
  1. > npm i - make order in node_modules
  2. > npm audit - see vulnerability issues
  3. > npm audit fix - fix vulnerability issues that don't require attention
  4. Create a pull-request fix-vulnerability-issues
  1. > npm i npm-check-updates -g - install npm-check-updates globally
  2. > npm-check-updates - see how outdated packages are
  3. > npm outdated - see how outdated packages are
  4. > npm update --save - update packages respecting your semver constraints from packages.json
  5. If you have packages that use major version 0.*.* you'll need to manually update these now
    • > npm install that-one-package@latest
  6. Create a pull-request update-packages-minor

If you're brave and can test/run you project easily:

  1. ncu -u - updates packages.json to all latest versions as shown by npm-check-updates
    • this might introduce breaking changes
  2. npm i - update package-lock.json
  3. Test your project.
  4. Create a pull-request update-packages-major

If you're not brave or can't just YOLO and update all major versions:

  1. npm-check-updates - check again what is left to update
  2. npm i that-package@latest - update major version of of that-package
  3. Test your project.
    • .js is dynamically typed so you might have just updated a package that breaks your project but you'll not know until you run your code
  4. Repeat for all packages.
  5. Create a pull-request update-packages-major

longer read

Need to update dependencies in a node js project? Here are my notes on this.

> npm i (npm install) 

> npm i

added 60 packages, removed 124 packages, changed 191 packages, and audited 522 packages in 13s

96 packages are looking for funding
  run `npm fund` for details

10 vulnerabilities (2 low, 7 moderate, 1 high)

To address issues that do not require attention, run:
  npm audit fix

To address all issues possible (including breaking changes), run:
  npm audit fix --force

Some issues need review, and may require choosing
a different dependency.

Run `npm audit` for details.
- installs missing packages in node_modules - removes redundant packages in node_modules - installs correct versions of mismatched packages (if packages-lock.json wants a different version than found in node_modules) - shows what is going on with packaged in your project

> npm audit - shows a report on vulnerability issues in your dependencies

> npm audit fix - updates packages to address vulnerability issues (updates that do not require attention)

> npm outdated - shows a table with your packages and versions 

$ npm outdated
Package      Current   Wanted   Latest  Location                  Depended by
glob          5.0.15   5.0.15    6.0.1  node_modules/glob         dependent-package-name
nothingness    0.0.3      git      git  node_modules/nothingness  dependent-package-name
npm            3.5.1    3.5.2    3.5.1  node_modules/npm          dependent-package-name
local-dev      0.0.3   linked   linked  local-dev                 dependent-package-name
once           1.3.2    1.3.3    1.3.3  node_modules/once         dependent-package-name

  • Current - what is in nodes_modules
  • Wanted - most recent version that respect the version constraint from packages.json
  • Latest - latest version from npm registry

To update to latest minor+patch versions of your dependencies (Wanted) - npm outdated shows all you need to know but I prefer the output of npm-check-updates

> npm i npm-check-updates -g (-g -> global mode - package will be available on your whole machine)

> npm-check-updates - shows where an update will be a major/minor/patch update (I like the colors) 

Checking C:\git\blog\package.json
[====================] 39/39 100%

 @azure/storage-blob         ^12.5.0  →      ^12.17.0
 adm-zip                     ^0.4.16  →       ^0.5.12
 axios                       ^0.27.2  →        ^1.6.8
 basic-ftp                    ^5.0.1  →        ^5.0.5
 cheerio                 ^1.0.0-rc.6  →  ^1.0.0-rc.12
 eslint                      ^8.12.0  →        ^9.2.0
 eslint-config-prettier       ^8.5.0  →        ^9.1.0
 eslint-plugin-import        ^2.25.4  →       ^2.29.1
 fast-xml-parser              ^4.2.4  →        ^4.3.6
 humanize-duration           ^3.27.3  →       ^3.32.0
 iconv                        ^3.0.0  →        ^3.0.1
 jsonwebtoken                 ^9.0.0  →        ^9.0.2
 luxon                        ^3.4.3  →        ^3.4.4

Let us update something

> npm update - perform updates respecting your semver constraints and update package-lock.json

> npm update --save - same as above but also update packages.json, use this one always

The behavior for packages with major version 0.*.* is different than for versions >=1.0.0 (see npm help update)

npm update will most likely bump all minor and patch versions for you.

You can run npm update --save often.

What do the symbols in package.json mean?

https://stackoverflow.com/questions/22343224/whats-the-difference-between-tilde-and-caret-in-package-json/25861938#25861938

npm update --save vs npm audit fix

npm audit fix will only update packages to fix vulnerability issues

npm update --save will update all packages it can (respecting semver constraints)

Do I have unused dependencies?

> npm install depcheck -g

> depcheck - shows unused dependencies. depcheck scans for require/import statements in your code so you might be utilizing a package differently but depcheck will consider it unused (ex. when you import packages using importLazy).

npm-check

> npm i npm-check -g

> npm-check - a different tool to help with dependencies (I didn't use it)

honorable mentions

> npm ls - list installed packages (from node_modules)

> npm ls axios - show all versions of axios and why we have them

npm ls will not show you origin of not-installed optional dependencies.

Consider this - you devleop on a win maching and deploy your solution to a linux box. On windows (see below) you might think node-gyp-build is not used in your solution. 

> npm ls node-gyp-build
test-npm@1.0.0 C:\git\test-npm
`-- (empty)

But on a linux box it will be used: 

> npm ls node-gyp-build
npm-test-proj@1.0.0 /git/npm-test-proj
└─┬ kafka-lz4-lite@1.0.5
  └─┬ piscina@3.2.0
    └─┬ nice-napi@1.0.2
      └── node-gyp-build@4.8.1

axios, cookies & more

axios

axios - promise-based HTTP client for node.js

  • when used in node.js axios uses http module (https://nodejs.org/api/http.html)
  • in node axios does not support cookies by itself (https://github.com/axios/axios/issues/5742)
    • there are npm packages that add cookies support to axios
  • when used in browsers it uses XMLHttpRequest
  • when used in browsers cookies work by default

Why would you use axios over plain http module from node?

Axios makes http requests much easier. Try using plain http and you'll convince your self.

Are there other packages like axios?

Yes - for example node-fetch https://github.com/node-fetch/node-fetch

When making a request axios creates a default http and https agent - https://axios-http.com/docs/req_config (axios probably uses global agents). You can specify custom agents for a specific request or set custom agents as default agents to use with an axios instance.

const a = require('axios');
const http = require('node:http');

(async () => {
    // configure your agent as needed
    const myCustomAgent = new http.Agent({ keepAlive: true });

    // use your custom agent for a specific request
    const x = await a.get('https://example.com/', { httpAgent: myCustomAgent });
    console.log(x);

    // set you agent as default for all requests
    a.default.httpAgent = myCustomAgent;
})();

What are http/s agents responsible for?

http/s agents handle creating/closing sockets, TCP, etc. They talk to the OS, manage connection to hosts.

cookies

Without extra packages you need to code reading response headers, look for Set-Cookie headers. Store cookies somewhere. Code adding cookie headers to subsequent request.

https://www.npmjs.com/package/http-cookie-agent

Manages cookies for node.js HTTP clients (e.g. Node.js global fetch, undici, axios, node-fetch). http-cookie-agent implements a http/s agent that inspects request headers and does cookie related magic for you. It uses the class CookieJar from package tough-cookie to parse&store cookies.

import axios from 'axios';
import { CookieJar } from 'tough-cookie';
import { HttpCookieAgent, HttpsCookieAgent } from 'http-cookie-agent/http';

const jar = new CookieJar();

const a = axios.create({
  httpAgent: new HttpCookieAgent({ cookies: { jar } }),
  httpsAgent: new HttpsCookieAgent({ cookies: { jar } }),
});
// now we have an axios instance supporting cookies
await a.get('https://example.com');

axios-cookiejar-support

https://www.npmjs.com/package/axios-cookiejar-support

Depends on http-cookie-agent and tough-cookie. Does the same as http-cookie-agent but you don't have to create http/s agents yourself. This is a small package that just intercepts axios requests and makes sure custom http/s agents are used source.

Saves you a bit of typing but you can't use your own custom agents. If you need to configure your http/s agents (ex. with a certificate) - use http-cookie-agent (see github issue and github issue)

import axios from 'axios';
import { wrapper } from 'axios-cookiejar-support';
import { CookieJar } from 'tough-cookie';

const jar = new CookieJar();
const client = wrapper(axios.create({ jar }));

await client.get('https://example.com');

https://www.npmjs.com/package/tough-cookie

npm package - cookie parsing/storage/retrieval (tough-cookie itself does nothing with http request).

A bit about cookies

https://datatracker.ietf.org/doc/html/rfc6265 - RFC describing cookies.

https://datatracker.ietf.org/doc/html/rfc6265#page-28 - concise paragraph on Third-party cookies.

Servers responds with a Set-Cookie header. Client can set the requested cookie. Cookies have a specific format described in this document.

Random stuff

https://npmtrends.com/cookie-vs-cookiejar-vs-cookies-vs-tough-cookie

interesting - cookie for servers are more popular than tough-cookie for clients since ~2023.

Is this due to more serve side apps being written in node?

Packages we don't use

  • cookie - npm package - cookies for servers
  • cookies - npm package - cookies for servers (different then cookie)
  • cookiejar - npm package - a different cookie jar for clients

fetch & fetch & node-fetch

fetch - standard created by WHATWG meant to replace XMLHttpRequest - https://fetch.spec.whatwg.org/

fetch - an old npm package to fetch web content - don't use it

node-fetch - community implemented fetch standard as a npm package - go ahead and use it

fetch - node's native implementation of the fetch standard - https://nodejs.org/dist/latest-v21.x/docs/api/globals.html#fetch

Since fetch standard is the standard for both browsers and node chrome has a neat feature to export requests to fetch

chat-gpt crap

When researching I came across some chat-gpt generated content. You read it thinking it will be something but it's trash.

https://www.dhiwise.com/post/managing-secure-cookies-via-axios-interceptors -> this article from 2024 that tell you to implement cookies your self, doesn't even mention the word "package", "module"

https://medium.com/@stheodorejohn/managing-cookies-with-axios-simplifying-cookie-based-authentication-911e53c23c8a -> doesn't mention that cookies don't work in axios run in node without extra packages (at least this one mentions that chat-gpt helped, thought I bet it's fully written by chat-gpt)

inco note - our http client is misleading, it uses same agent for http and https, it should maybe be called customAgent

axios and fiddler

Using a request interceptor (proxy) like fiddler helps during development and debugging.

To make fiddler intercept axios request we have to tell axios that there is a proxy where all requests from should go. The proxy forwards those requests to the actual destination.

http_proxy=... // set proxy for http requests
https_proxy=... // set proxy for https requests
no_proxy=domain1.com,domain2.com // comma separated list of domains that should not be proxied

The proxy for both http and https can be the same url.

Read more - https://axios-http.com/docs/req_config

When using fiddler on windows I suggest going to Network & internet > Proxy and disableing proxies there (fiddler by default sets this). This way fiddler will only receive requests from the process where we set http(s)_proxy env vars.

fiddler and client certificates

I was not able to make fiddler work with client certificates. It should be done like this - https://docs.telerik.com/fiddler/configure-fiddler/tasks/respondwithclientcert but I couldn't get it to work

honorable mentions

I would like to try out - https://www.npmjs.com/package/proxy-agent at some point

I don't fully understand withCredentials

axios & cookies demo

> npm i
> node server.mjs
open browser and go to http://127.0.0.1:3000
cookies are supported
> node test.js (from another console)
cookies are not supported

axios, certificates, etc

To use axios with a client certificate you need to configure the https agent with the key and cert. the key and cert need to be in pem format. They both can be in the same pem file, or in separate pem files. (did not try it) but you should be able to merge and split your pem.

https://nodejs.org/api/tls.html#tlscreatesecurecontextoptions

to try out - https://www.npmjs.com/package/proxy-agent