JavaScript has a number of predefined objects. In addition, you can create your own objects.

Using object initializers#

It is sometimes referred to as creating objects with literal syntax. It’s one of easiest way to create an object by simply define the property and values inside curly braces.

let product = {
  mfgCountry: 'Indonesia'
};
product.name = 'Surgical Mask';
product.category = 'medical';

Using Object class#

let product = new Object(); // store an empty Object object in product.
product.name = 'Surgical Mask';
product.category = 'medical';
product.mfgCountry = 'Indonesia';

Using Object’s create method#

// Create a new object using an existing object as the prototype.
let product = Object.create({
  name: this.name,
  category: this.category,
  mfgCountry: 'Indonesia'
}, {
  name: { value: 'Surgical Mask' },
  category: { value: 'medical' },
});

Using Object.prototype.constructor function#

function Product(name, category) {
  this.name = name;
  this.category = category;
  this.mfgCountry = 'Indonesia';
}
let product = new Product('Surgical Mask', 'medical');
console.log('product.constructor is ' + product.constructor);

Using ES6 Class syntax#

JavaScript classes are primarily syntactical sugar over JavaScript’s existing prototype-based inheritance.

class Product {
  constructor(name, category) {
    this.name = name;
    this.category = category;
    this.mfgCountry = 'Indonesia'
  }
}
let product = new Product('Surgical Mask', 'medical');

Using Immediately Invoked Function Expression (IIFE)#

IIFE is a design pattern which is also known as a self-executing anonymous function.

let product = new(function (name, category) {
  this.name = name;
  this.category = category;
  this.mfgCountry = 'Indonesia';
})('Surgical Mask', 'medical');

Using Function constructor with prototype#

(Note: this is NOT one of the way to create a JavaScript object with the same behavior as the rest of the examples in this post because it set the property on the prototype and not the object it self.)

function Product() {
  this.mfgCountry = 'Indonesia';
}
Product.prototype.name = 'Surgical Mask';
Product.prototype.category = 'medical';
let product = new Product();

I deliberately show this example because this is where a lot of confusion usually begins with JavaScript prototypes.

Deno is a secure runtime for JavaScript and TypeScript.

Deno is a brand new general-purpose JavaScript programming environment, similar to Node.js. It is what Node.js should have been according to Ryan Dahl who created both tools.

Deno just hit v1.0.0-rc1. David Else has written a nice article about Deno 1.0: what you need to know.

This guide should give you something you need to get started.

Getting Started#

• Running a simple Hello World script
• Writing our own script
• Using Deno with TypeScript

Installation#

Full guide for installing Deno for other platforms.

Using shell (macOS and Linux):

$ curl -fsSL https://deno.land/x/install/install.sh | sh
######################################################################## 100.0%##O=#  #                                                                      
Archive:  /home/cedric/.deno/bin/deno.zip
  inflating: deno                    
Deno was installed successfully to /home/cedric/.deno/bin/deno
Manually add the directory to your $HOME/.bash_profile (or similar)
  export DENO_INSTALL="/home/cedric/.deno"
  export PATH="$DENO_INSTALL/bin:$PATH"
Run '/home/cedric/.deno/bin/deno --help' to get started

Example - Running a simple “Hello World” script#

Try running a simple program:

$ deno run https://deno.land/std/examples/welcome.ts
Download https://deno.land/std/examples/welcome.ts
Compile https://deno.land/std/examples/welcome.ts
Welcome to Deno 🦕

Deno tries to be web compatible and use modern features whereever possible.

Writing our own script#

Next, we will go through some simple examples that can teach you about the fundamentals of Deno.

Web server#

$ vim web_server.ts

Copy and paste the following code into your text editor.

import { serve } from "https://deno.land/std@v0.42.0/http/server.ts";
const s = serve({ port: 8000 });
console.log("http://localhost:8000/");
for await (const req of s) {
  req.respond({ body: "Hello world\n" });
}

Try the program:

$ deno run web_server.ts
Compile file:///home/cedric/my_files/deno-test/web_server.ts
Download https://deno.land/std@v0.42.0/http/server.ts
Download https://deno.land/std@v0.42.0/encoding/utf8.ts
Download https://deno.land/std@v0.42.0/io/bufio.ts
Download https://deno.land/std@v0.42.0/testing/asserts.ts
Download https://deno.land/std@v0.42.0/util/async.ts
Download https://deno.land/std@v0.42.0/http/io.ts
Download https://deno.land/std@v0.42.0/io/util.ts
Download https://deno.land/std@v0.42.0/path/mod.ts
Download https://deno.land/std@v0.42.0/path/win32.ts
Download https://deno.land/std@v0.42.0/path/posix.ts
Download https://deno.land/std@v0.42.0/path/constants.ts
Download https://deno.land/std@v0.42.0/path/common.ts
Download https://deno.land/std@v0.42.0/path/constants.ts
Download https://deno.land/std@v0.42.0/path/interface.ts
Download https://deno.land/std@v0.42.0/path/glob.ts
Download https://deno.land/std@v0.42.0/path/globrex.ts
Download https://deno.land/std@v0.42.0/path/utils.ts
Download https://deno.land/std@v0.42.0/fmt/colors.ts
Download https://deno.land/std@v0.42.0/testing/diff.ts
Download https://deno.land/std@v0.42.0/textproto/mod.ts
Download https://deno.land/std@v0.42.0/http/http_status.ts
Download https://deno.land/std@v0.42.0/bytes/mod.ts
error: Uncaught PermissionDenied: network access to "127.0.0.1:8000", run again with the --allow-net flag
    at unwrapResponse ($deno$/ops/dispatch_json.ts:43:11)
    at Object.sendSync ($deno$/ops/dispatch_json.ts:72:10)
    at Object.listen ($deno$/ops/net.ts:51:10)
    at listen ($deno$/net.ts:152:22)
    at serve (server.ts:261:20)
    at web_server.ts:2:11

You will see that this program returns an error regarding network access because Deno is a runtime that is secure by default.

$ deno run --allow-net web_server.ts
http://localhost:8000/

$ curl http://localhost:8000
Hello world

Making an HTTP request#

A small program that fetches a file from a web server and prints the content to the terminal.

// curl.ts

const url = Deno.args[0];
const res = await fetch(url);

const body = new Uint8Array(await res.arrayBuffer());
await Deno.stdout.write(body);

Just like in the browser you can use the web standard fetch API to make HTTP calls.

This program parse the response body as an ArrayBuffer, await the response, convert it into a Uint8Array and store it in the variable body.

$ deno run --allow-net curl.ts https://example.com
Compile file:///home/cedric/my_files/go-dist-services/curl.ts
<!doctype html>
<html>
<head>
    <title>Example Domain</title>

    <meta charset="utf-8" />

... truncated ...
... truncated ...

Reading a file#

Deno also provides APIs which do not come from the web. You can find documentation for these APIs on doc.deno.land.

Filesystem APIs for example do not have a web standard form, so Deno provides its own API.

// cat.ts

for (let i = 0; i < Deno.args.length; i++) {
  let filename = Deno.args[i];
  let file = await Deno.open(filename);
  await Deno.copy(file, Deno.stdout);
  file.close();
}

Try running the program:

$ deno run --allow-read cat.ts hello.txt
Hello from Deno.

This illustrates a general design goal for I/O streams in Deno.

A simple TCP server#

An example of a simple server which accepts connections on port 8080, and returns to the client anything it sends.

// echo_server.ts

const listener = Deno.listen({ port: 8080 });
console.log("listening on 0.0.0.0:8080");
for await (const conn of listener) {
  Deno.copy(conn, conn);
}

Try:

$ deno run --allow-net echo_server.ts
listening on 0.0.0.0:8080

Try sending data to it with netcat:

$ nc localhost 8080
hello deno
hello deno

Like the cat.ts example, the copy() function here also does not make unnecessary memory copies. It receives a packet from the kernel and sends back, without further complexity.

Using TypeScript#

Please read the docs.

Learn more#

• Deno in 100 Seconds (video).
• 10 Things I Regret About Node.js - Ryan Dahl - JSConf EU (2018, video).
  • History of Deno. Regrets: not sticking with Promises, security, the build system (GYP), package.json, node_modules, require(“module”) without the extension “.js”, index.js
• Talk from Bert Belder (@piscisaureus) - A decade after Node.js was first announced, JavaScript and the web platform have evolved, and TypeScript has changed the way JavaScript applications are developed.

I’ll start with a little bit of background.

So, I’ve developed several Single Page Applications (SPAs) using JavaScript over many years. These are a few picks of “large” JavaScript apps in production:

• Pseudo 3D mobile racing game - build with Phaser framework
• Hybrid mobile app - build with Apache Cordova (was PhoneGap) and jQuery
• Integrated DICOM medical images viewer with accelerated anomaly detection backed by deep learning - build with React and Electron

I’ve also developed MVC web applications using RoR or Node.js. The view layer (UI) is server rendered.

Being on both sides, one of the pain point of developing SPAs using JavaScript library like React or Vue.js is, it is difficult to have good SEO if you are using client-side rendering (CSR) in React for your public facing website (like marketing site, blog, documentation site, content-heavy site like news and media). Yes, I am aware that Googlebot (web crawler) processes JavaScript.

There are a number of different ways to build a website. To solve this pain point (along with other problems with CSR), we use different techniques like Server-side rendering (SSR), prerendering, and so on.

To get the best from both worlds, I pick Next.js for some problems. Next.js is the React framework. Next.js render the HTML for a page on the server in response to navigation. Next.js is in the category of SSR with rehydration. Rehydration is “booting up” JavaScript views on the client such that they reuse the server-rendered HTML’s DOM tree and data.

It’s hard to get React SSR to work properly. This is where we see different React frameworks come into play.

In the same category of frameworks like Next.js is Remix (Cool site! And one more thing, it’s designed for Web Accessibility).

Finally, a killer React framework from the creators of React Router

Remix is a new React framework that provides APIs and conventions for server rendering, data loading, routing and more.

Check out the first preview of Remix. Don’t miss the video walkthrough in that post. You get to see:

… the best way to build dynamic React websites. You’ll get a preview on routing, layouts, data loading, meta tags, data caching, and scroll restoration.

In contrast to Next.js, IMO, if there’s one thing I felt clunky in Next.js and that is its router. It’s interesting to see how Remix approach this, how will this turns out eventually.

Alright, that’s all I have for you in this post. I hope you enjoy it. You can follow me for more such interesting stories on Twitter.

Hey, one more thing before you go, have you seen RedwoodJS or Blitz?

This is part 1 of a series of upcoming posts on my career and code retreat retrospective — what has been great, what has been horrible. Please pardon my English as I am not a native English speaker.

What is Educational/Code Retreat#

I was excited to first learned about the idea of educational retreat from Julia Evans’ blog in 2015. She attended Hacker School. In the words of the Recurse Center (formerly known as Hacker School) website:

The Recurse Center is a free, self-directed, educational retreat for people who want to get better at programming, whether they’ve been coding for three decades or three months.

It’s as a retreat for programmers who want to get back into learning. You take sabbatical from work, away from deadlines and the stresses of everyday life. It’s sort of like going back to college, but you get to decide what you want to learn, and homework is never boring because you get to choose the fun stuff that you want to work on.

My Experience#

I spent 14 months in educational retreat from 2019 to Feb 2020 — 7 months as an Entrepreneur-In-Residence (EIR) at Antler and the rest of the time in code retreat ran by a startup in Asia. I’ve been meaning to write about the experience itself, and what happened after for a while now, so here goes!

The Journey#

When I moved to Singapore in 2006, my main objective is to start my own tech startup business and chase my dream. I work hard and grab every little opportunities offered to me that allow me to learn and grow my skills and career while working for companies. I am always hungry to learn. Learning never stops. I still remember vividly my first ever experience attending Barcamp 4 conference in 2009.

Career as a Full Stack Software Engineer#

I am a full stack software engineer all along (funnily, even before full stack title being a thing now). I mostly work on web developments in the early days of my career. I learned iOS (and Objective-C)¹ and Android (Java) mobile development not long after the first iPhone and AppStore launched together with a group of people who started CocoaHeads meetup, buUuk and some worked at Apple now. I’ve blogged about the iPhone 3G launch and my first foray into Android development in 2010 after attended the Developer Lab by Google in my abandon blogs.

Front-end Development#

In my full stack web roles, I spent most of my time on front-end development, writing vanilla JavaScript, CSS, jQuery, Backbone, and a bit of MooTools. I picked up React.js a year after it was announced as open source project in 2013. I switched to React Native in 2015 (video) after working in PhoneGap. Since then, I have delivered over 5 major React projects to production, be it for my consulting customers or my own startups.

Backend Development#

Most of my experience in backend works are done in statically typed and compiled languages like C, C# or TypeScript. I have a short love with Ruby before Node.js came out. I have only done some toy projects with Ruby on Rails and that was long time ago. I have deep experience in Node.js together with a minimalistic MVC framework, Express.

I don’t have many achievements in backend development that I’m proud of. I think my favorite contributions to backend systems were my work optimizing SQL database performance for several large legacy websites, created my first in-house web framework from scratch, RESTful API design and implementation for small to medium-sized apps, and sysadmin lead for greenfield infra projects on AWS.

As you can see, I lack the experience and skills in working in high traffic and large apps where majestic monolith/macroservices (opposite of micro/services oriented architecture) runs in a distributed systems environment and learning to build secure, reliable, and maintainable systems are important skills.

More than Just Programming#

If you ask me to describe “what is software engineering in three words?”, my response is “more than programming”.

Throughout my time as a software engineer and consultant, I have lead technically several major projects and supported customers for both business and technical needs. My team are structured to work directly and closely with the customers in every project. This will minimize the cost of error due to communication gap. We wear many hats. We go beyond our own roles sometimes.

Mid-Career Switch#

Around 2013, I noticed my career growth stagnant even though I still learn every day on the job, but not much. So, I started making a change to the situation. At this point, I have learned enough the basics of building startup and high-growth business after getting to know entrepreneur27 (e27 now) and TechCrunch.

First Taste of Deep Learning#

In 2015, I was confronted with a problem that is not feasible to solve using existing way of manually hand-written programming. The problem is simple, such that, “set the web font color automatically while maintaining good contrast with its background color”.

As a curious learner and knowing the chance to step-up my learning several times, I spent a week exploring and researching for solution and eventually ended up discovering neural networks. As a self-taught Computer Science, I have no idea what is neural networks. But that didn’t stop me from learning.

Then, I stumbled upon Stanford CS231n MOOC taught by Andrej Karpathy and started learning deep learning, convnet, and Computer Vision. I was still struggling to wrap my mind around all the new neural nets theory and concepts and Maths around backpropagation/gradient descent calculations. So, I persisted and studied Coursera Machine Learning course taught by Prof. Andrew Ng and got certified.

The Growth Stage#

I was doing all that while having a full-time job. But, at the end of 2017, I left my 6 years job and pursue my dream. It’s my first sabbatical. I build my first startup using what I learned so far.

While working on my first startup in the medical domain, from 2018 onwards, I started my fellowship at fast.ai. Unlike Stanford’s courses, fast.ai teaching style is geared towards practical examples first, theory later and top-down learning (I get to play baseball before learning about the sport science). fast.ai courses complement well with the more formal courses from Stanford and Coursera. During this period, I learned immensely up to the point I am capable of moving beyond deep learning research and experimentation and taking real-world projects to production in my startup. Unfortunately, my startup failed to gain traction at the end of 2018 and demised.

Fast.ai International Fellowship#

I was accepted into Fast.ai International Fellowship 2018 program. As a fellow, I worked on a variety of data science and deep learning projects such as:

• Awesome BERT NLP repo - I have been tracking and researching Transformers networks and transfer learning in NLP for more than a year. Along the way, I share and update the repo with stuffs I find awesome.
• I wrote a Deep Learning for Coders book based on my learnings and notes that I took while studying Stanford CS231 CNN, Stanford CS224 NLP, Coursera Machine Learning, and fast.ai courses.
• Data science Python notebooks - a collection of Jupyter notebooks on machine learning and deep learning. My favorites are “Language modelling in Malay language for downstream NLP tasks”, “Not Hotdog AI Camera mobile app”, “Deep Painterly Harmonization”, and “Guide to TensorFlow + Keras on TPU v2 for free on Google Colab”.
• PyTorch Mobile Kit is a starter kit app that does Machine Learning on edge from camera output, photos, and videos.
• Fast.ai Mobile Camera is a guide and demo on “How I Shipped a Neural Network on Android/iOS Phones with PyTorch and Android Studio/Xcode”. The Android app is doing real time object classification. [blog post from my team]
• PyTorch CapsNet is the first few PyTorch implementation of Capsule Network NIPS 2017 paper by Geoffrey Hinton, et al. This is my first time implementing deep learning research paper from scratch using PyTorch 0.3.0.

First Retreat#

My first educational retreat was in early 2019. I applied and got accepted out of 3000+ applicants in SEA to join Antler as an Entrepreneur-In-Residence (EIR). It’s sort of apprentice program and business retreat for entrepreneurs building startup. Our team failed to secure a pre-seed funding from Antler and my journey as an entrepreneur ended there. This itself is a blog post on its own, next time.

I wasn’t getting hired as a Data Scientist#

After Antler, I went back to my original plan which is, I was trying to switch careers and land a new job as deep learning engineer or data scientist. I failed to break into this role after spending months interviewing. I got rejected multiple times because companies are either not confident in hiring someone like me without a Machine Learning/Deep Learning PhD or someone that cannot recite backpropagation algorithms during coding interview sessions.

Instead of focusing on skills thought to be required of data scientists, we can look at what they have actually done before. (source)

This is the truth?

I know hiring is broken for SWE roles at big tech companies but I never expect this bad. OK, I will stop there. This is not a post to rant about hiring is broken.

To add on, I have not touched data science for a while. Why? In my opinion, if you are not FAANG, it’s costly to scale data science to production grade where your models can work robustly under known adversarial attacks. Data/AI ethics are challenging. Another smaller issue is research is not catching up fast enough with business demands of interpretable models. A lot more has been said on those issues. Check out “Rebooting AI” by Gary Marcus. Another example is this “Data Science: Reality Doesn’t Meet Expectations” post.

I’m sorry to say that I don’t have solutions now. I will consider to give data science another go in the future.

Why did you join code retreat?#

I found out about RC in a period of my career when I wasn’t growing much. I was frequently the sole software engineer on my team and frequently worked on projects on my own.

The Second Code Retreat#

I’ve moved on from pursuing Data Scientist role. So I started my journey into the second educational retreat phase. The startup company that runs the code retreat is flexible enough that allows us to design and customize plan for our retreat. My design mainly drew inspirations from Recurse Center User’s Manual and my experience organizing and running virtual/in-person study groups for fast.ai and AI Saturdays students. I also referenced a few online articles:

• Example of how Recurser spent their time at RC.
• Why I Decided to Learn to Code in Bali with Institute of Code
• Coderetreat was spawned by Gary Bernhardt, Corey Haines, and many more.

The goal and focus of this code retreat is I want to get significantly better at programming and the fundamentals of Computer Science. As Richard Feynman put it:

What I cannot create, I do not understand

And I want to be a wizard :smile: Joke aside, during the COVID-19 crisis, where most of us are stuck at home and work suspended, this is the best time to start doing things you always wanted to do.

What did you do at your code retreat?#

I worked on a variety of projects. Some reached completion, and some didn’t. I did things that I’ve always wanted to do, but also ended up doing things I never thought I would. Some of my favorites (marked as *) are:

• * Learned a new programming language, Rust. I created a very inaccurate “knowledge map” of stuffs I referenced learning Rust.
• Learned to make my own version of the classic command line tool grep using Rust.
• Build a basic multithreaded web server from scratch (zero dependencies) using Rust and along the way, learn a bit about TCP and HTTP.
• * Implements Conway’s Game of Life in Rust and WebAssembly.
• Learn Rust by creating too many linked lists.
• * KVS, a high-performance, networked, parallel and asynchronous key/value store (embedded database like SQLite) developed in Rust.
• RNNoise is a Rust and Node.js bindings to Xiph’s RNNoise denoising C library.
• * Deep dive into Xi Editor internals. Xi is a modern editor for the next 20 years. This exercise altered my appreciation for persistent rope data structure.
• * You Don’t Know Go, Yet book is a project for me to relearn Go from first principles by writing a book on it. I learned advanced Go stuffs in this study.
• Learn Test-Driven Development (TDD) in Go for the last time.
• * MinTorrent is yet another minimalistic, zero dependency torrent client, written in Go (golang).
• * Build yourself a programming language. I created Hou programming language interpreter and compiler from scratch using Go.
• Snippetbox is a project I created to learn full-stack Go web application development.
• * Learn distributed systems from first principles by doing MIT 6.824 distributed system labs and reading Designing Data Intensive Applications book (learning-in-progress). I’m currently reading academic papers and reimplementing Raft consensus algorithm.
• …and more (are you here and looking for project ideas to practice a new programming language?)

A lot of them are focused on Computer Science, systems or Linux concepts and go deep into the concepts.

I will continue in another post going into much more detail on each project.

Lastly, I would to take this opportunity to apologize to anyone who have contacted me and I didn’t response back. It is not in my truest intention as I was beating and recovering from burnout last year.

Hire Cedric Chee#

I’m back on the job market! I just lost my freelance job, and I was living without a stable income for the past 9 months.

Contact info and details here.

That’s it for now. Thank you and take care.

Job Hunt

It’s Saturday night here. Tonight, I have a chance to revisit some articles in my reading list. The topic is, “using shiny, untested technology”. Here are a few of my selections:

1. The Magpie Developer by Jeff Atwood in 2008.

we are too easily distracted by shiny new toys and playthings.

2. The classic article “Why You Should Never Use MongoDB” by Sarah Mei in 2013.

• I’ve reread the article in 2020 to remind myself again. Sarah Mei is a thought leader in our industry. This article is gold. The thoughts there have somehow guided me in making some decisions in that era. For example, choosing between SQL or NoSQL database for my past web development works.

3. Choose boring technology by Dan McKinley.

• The innovation tokens concept is interesting.

Every time I spin up a new project, I try to answer the following question honestly:

“Am I using this project as an excuse to learn some new technology, or am I trying to solve a problem?”

— aaronbrethorst on HN

4. Boring technology is awesome by John Hyland.

Summary: There are a few universal problems with new technologies that you can count on:

• APIs can change out from under you.
• They’re less secure.
• There’ll be fewer established standards.
• They lack basic tooling.

When to use shiny tech

• Side projects
• Internal tooling
• When it’s worth it

How to handle the hotness (if you must)

• Roll it out slowly.
• Standardize your usage of the new technology.

5. The boring technology behind a one-person Internet company by Wenbin Fang

• Technology is usually just a means to an end. I’ve seen so many teams burn so much energy on complicated stacks just to drink kool-aid.

6. Enough with the microservices by Adam Drake.

“Much has been written on the pros and cons of microservices, but unfortunately I’m still seeing them as something being pursued in a cargo cult fashion in the growth-stage startup world.”

Don’t even consider microservices unless you have a system that’s too complex to manage as a monolith. The majority of software systems should be built as a single monolithic application. Do pay attention to good modularity within that monolith, but don’t try to separate it into separate services.

– Martin Fowler

If you can’t build a well-structured monolith, what makes you think microservices is the answer?

– Simon Brown

You are not FAANG (Facebook, Apple, Amazon, Netflix, Google), stop trying to be them!

7. “Let’s use Kubernetes!” Now you have 8 problems by Itamar Turner-Trauring.

8. 4 engineering mistakes that kill startups

• “From chasing shiny tech to scaling too soon, beware these common missteps that can endanger your company”.

9. Hype Driven Development by Marek Kirejczyk.

10. Thinking The Impact Of Technical Decision by Didiet Noor.

What is the solution?

In my own opinion, focus on your fundamentals. Understand that, if your foundations aren’t strong and your fundamentals are not clear, then no matter what shiny new things you try to learn, it won’t help you for a long run. So, don’t ignore basic concepts before rushing to any new and shiny technologies.

Thank you for reading.

Happy weekend!

Long before, I have this theory: Go compiler is so fast that we can use it like an “interpreted” scripting language.

Yesterday, this wild idea surfaced again when I stumbled upon this Reddit post and discussion:

https://www.reddit.com/r/golang/comments/fcumfu/golang_is_my_new_shell/

Last year, I saw some early attempts like the one in this post by Eyal in 2017:

Story: Writing Scripts with Go.

However, I stop there as I felt the approach is a bit clumsy. And no, I don’t expect Go Authors to officially bake this support in Go tooling.

The post from Reddit yesterday really pique my interest again. So, I continue reading and discovered this great blog post by Cloudflare (2018):

Using Go as a scripting language in Linux.

Next, I proceed to try it out and I found the approach is nearly perfect for my use case.

So, along the way, I learned how Linux executes files. By reading the Linux kernel documentation, kernel support for miscellaneous (your favourite) binary formats.

The binfmt_misc is a script module responsible for parsing shebang lines and executing scripts on the target system.

Who knows that the shebang support is actually implemented in the kernel itself and not in the shell or other daemon/process? At least not me.

Once in a while, I like fiddling with some kernel ‘magic’. Today is the day.

As I think back of today’s tiny experiment, it reminds me again of what important things I’ve forgotten.

One way for me to be a better software engineer, be it for front-end or back-end engineering is to learn the fundamentals (of computer science). That is, to me is the Operating System (OS) like Unix or Linux. Today’s example tells me that almost every time I go back and learn about the Linux kernel, it taught me new things that surprises me.

I suggest, if you have not, you should check The Linux kernel documentation. There are many hidden gems there, like the one I encountered today.

That’s it for today. Thanks for reading.

Have a nice day.

I’m learning WebAssemby with Rust and Emscripten.

First, download and install Rust:

cedric@trinine:~/cedric/learn-rust$ curl https://sh.rustup.rs -sSf | sh
info: downloading installer

Welcome to Rust!

This will download and install the official compiler for the Rust programming
language, and its package manager, Cargo.

It will add the cargo, rustc, rustup and other commands to Cargo's bin
directory, located at:

/home/cedric/.cargo/bin

This path will then be added to your PATH environment variable by modifying the
profile file located at:

/home/cedric/.profile

You can uninstall at any time with rustup self uninstall and these changes will
be reverted.

Current installation options:

    default host triple: x86_64-unknown-linux-gnu
    default toolchain: stable
modify PATH variable: yes

1) Proceed with installation (default)
2) Customize installation
3) Cancel installation
>

info: syncing channel updates for 'stable-x86_64-unknown-linux-gnu'
info: latest update on 2019-07-04, rust version 1.36.0 (a53f9df32 2019-07-03)
info: downloading component 'rustc'
info: downloading component 'rust-std'
info: downloading component 'cargo'
info: downloading component 'rust-docs'
info: installing component 'rustc'
91.1 MiB /  91.1 MiB (100 %)  13.8 MiB/s in  6s ETA:  0s
info: installing component 'rust-std'
61.3 MiB /  61.3 MiB (100 %)  16.1 MiB/s in  3s ETA:  0s
info: installing component 'cargo'
info: installing component 'rust-docs'
11.0 MiB /  11.0 MiB (100 %)   6.8 MiB/s in  1s ETA:  0s
info: default toolchain set to 'stable'

stable installed - rustc 1.36.0 (a53f9df32 2019-07-03)


Rust is installed now. Great!

To get started you need Cargo's bin directory ($HOME/.cargo/bin) in your PATH
environment variable. Next time you log in this will be done automatically.

To configure your current shell run source $HOME/.cargo/env

WebAssembly Tutorial#

This section will show you how to build and run your first Rust and WebAssembly program: a Web page that alerts “Hello, World!”

You can follow along the original tutorial here.

cedric@trinine:~/cedric/learn-rust/wasm-game-of-life$ wasm-pack build
[INFO]: Checking for the Wasm target...
info: downloading component 'rust-std' for 'wasm32-unknown-unknown'
info: installing component 'rust-std' for 'wasm32-unknown-unknown'
[INFO]: Compiling to Wasm...
   Compiling proc-macro2 v1.0.1
   Compiling unicode-xid v0.2.0
   Compiling log v0.4.8
   Compiling syn v1.0.3
   Compiling wasm-bindgen-shared v0.2.50
   Compiling cfg-if v0.1.9
   Compiling bumpalo v2.6.0
   Compiling lazy_static v1.3.0
   Compiling wasm-bindgen v0.2.50
   Compiling quote v1.0.2
   Compiling wasm-bindgen-backend v0.2.50
   Compiling wasm-bindgen-macro-support v0.2.50
   Compiling wasm-bindgen-macro v0.2.50
   Compiling console_error_panic_hook v0.1.6
   Compiling wasm-game-of-life v0.1.0 (/home/cedric/learn-rust/wasm-game-of-life)
warning: function is never used: `set_panic_hook`
 --> src/utils.rs:1:1
  |
1 | pub fn set_panic_hook() {
  | ^^^^^^^^^^^^^^^^^^^^^^^
  |
  = note: #[warn(dead_code)] on by default

    Finished release [optimized] target(s) in 47.93s
[INFO]: Installing wasm-bindgen...
[INFO]: Optional fields missing from Cargo.toml: 'description', 'repository', and 'license'. These are not necessary, but recommended
[INFO]: :-) Done in 52.62s
[INFO]: :-) Your wasm pkg is ready to publish at ./pkg.

$ npm init wasm-app www
npx: installed 1 in 2.302s
🦀 Rust + 🕸 Wasm = ❤

Publishing to NPM:

cedric@trinine:~/cedric/learn-rust/wasm-game-of-life$ wasm-pack login
Username: *********
Password:
Email: (this IS public) ********
npm ERR! code E401
npm ERR! Incorrect or missing password.
npm ERR! If you were trying to login, change your password, create an
npm ERR! authentication token or enable two-factor authentication then
npm ERR! that means you likely typed your password in incorrectly.
npm ERR! Please try again, or recover your password at:
npm ERR!     https://www.npmjs.com/forgot
npm ERR!
npm ERR! If you were doing some other operation then your saved credentials are
npm ERR! probably out of date. To correct this please try logging in again with:
cedric@trinine:~/cedric/learn-rust/wasm-game-of-life$

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/cedric/.npm/_logs/2019-08-22T11_46_05_314Z-debug.log
Error: Login to registry https://registry.npmjs.org/ failed

# Retry step

$ wasm-pack login
Username: ********
Password:
Email: (this IS public) ***************
Logged in as ********* on https://registry.npmjs.org/.
[INFO]: 👋  logged you in!

cedric@trinine:~/cedric/learn-rust/wasm-game-of-life$ wasm-pack publish
npm notice
npm notice 📦  wasm-game-of-life@0.1.0
npm notice === Tarball Contents ===
npm notice 301B   package.json
npm notice 2.2kB  README.md
npm notice 15.5kB wasm_game_of_life_bg.wasm
npm notice 772B   wasm_game_of_life.d.ts
npm notice 2.8kB  wasm_game_of_life.js
npm notice === Tarball Details ===
npm notice name:          wasm-game-of-life
npm notice version:       0.1.0
npm notice package size:  9.4 kB
npm notice unpacked size: 21.5 kB
npm notice shasum:        f0acf58e757f73b8a615a9b6bb7720823dc6ddad
npm notice integrity:     sha512-hl+J9N+gKAch/[...]hLQJHNyRb/zAQ==
npm notice total files:   5
npm notice
This operation requires a one-time password.
Enter OTP: *********
npm ERR! code E403
npm ERR! 403 Forbidden - PUT https://registry.npmjs.org/wasm-game-of-life - Package name too similar to existing packages; try renaming your package to '@scope/wasm-game-of-life' and publishing with 'npm publish --access=public' instead

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/cedric/.npm/_logs/2019-08-22T13_13_42_232Z-debug.log
Error: Publishing to npm failed
Caused by: failed to execute `npm publish`: exited with exit code: 1

To learn more, you can take a look at the learning resources below:

Examples#

Recent example of projects:

• petersalomonsen/wasm-git - Git for Node.js and the browser using https://libgit2.org compiled to WebAssembly with Emscripten

Tools and Libraries#

• wasmtime - A small, efficient, and standalone JIT-style runtime for WebAssembly and WASI, using Cranelift.

I’m reading the recently published JPEG XL spec.

Brunsli lossless recompression system and FUIF responsive image is interesting.

So, I go on an exploration and learned a few things about JPEG XL.

• Brunsli: Practical JPEG repacker
  • Brunsli is a lossless JPEG recompression system library.
  • Brunsli allows for a 22% decrease in file size while allowing the original JPEG to be recovered byte-by-byte.
  • GOOD NEWS: now part of JPEG XL draft
• JPEG XL
  • Brunsli has been specified as the lossless JPEG transport layer in the Committee Draft of JPEG XL Image Coding System.
  • Ready to power faster and more economical transfer and storage of photographs.
  • Overview of JPEG XL
• ImageMagick
  • ImageMagick adding support for JPEG XL
  • ImageMagick is the most central “Swiss Army Knife” for imaging in the web ecosystem.
• From ImageCon 2019: “JPEG XL: Next-Generation of Image Format for the Internet” [video]
  • Presentation slides by Jon Sneyer: https://www.slideshare.net/cloudinarymarketing/imagecon-2019-jon-sneyer
  • Some context: JPEG XL is a combination of Cloudinary’s FUIF (successor of FLIF) and Google’s Pik.
    • Great HN discussion on FLIF
• Benchmark/comparison
  • A simple comparison between AV1, BPG, WebP and Pik for use in image compression (in Japanese language).
    • Pik won it. JPEG XL is an improved from Pik.
• Demo
  • Brunsli demo of JPEG XL lossless recompression .jpg ⇄ .jxl
• Tools
  • JPEGXL - JavaScript - you can encode to JPEG XL here. They compiled C/C++ code to WebAssembly.

We care for making the Internet faster.

Future of image compression with utmost respect to our digital heritage.

My take: I consider Brunsli a very promising approach as the successor of JPEG standard.

A random and wild adventure into the WireGuard land.

What is WireGuard?

Is it the replacement of OpenVPN, IPSec? Is it the future of VPN. I don’t want to do injustice to WireGuard. I think WireGuard website explained it pretty well, even for not very technical people.

Installation for Ubuntu 16.04#

You can see the full glory in the logs/terminal output of this adventure. If you pay enough attention, you will notice that I banged my head trying to get WireGuard kernel module works. To save you from repeating my mistakes, do remember where (which environment: virtualized, VM, container or host) you run your commands; DO NOT install Linux kernel while you are inside Docker container.

Step 0:

• Check if a process is running inside docker container
• If you are installing and using Wireguard with containers, please refer to this blog post and instructions here: https://blog.jessfraz.com/post/installing-and-using-wireguard/

Step 1:

Steps and instructions.

1. Install and setup WireGuard kernel module:

$ sudo add-apt-repository ppa:wireguard/wireguard

$ sudo apt-get update

$ sudo apt install wireguard

Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  wireguard-dkms wireguard-tools
The following NEW packages will be installed:
  wireguard wireguard-dkms wireguard-tools
0 upgraded, 3 newly installed, 0 to remove and 2 not upgraded.
Need to get 358 kB of archives.
After this operation, 2,042 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://ppa.launchpad.net/wireguard/wireguard/ubuntu xenial/main amd64 wireguard-dkms all 0.0.20200215-0ppa1~16.04 [259 kB]
Get:2 http://ppa.launchpad.net/wireguard/wireguard/ubuntu xenial/main amd64 wireguard-tools amd64 1.0.20200206-0ppa1~16.04 [91.6 kB]
Get:3 http://ppa.launchpad.net/wireguard/wireguard/ubuntu xenial/main amd64 wireguard all 1.0.20200206-0ppa1~16.04 [7,334 B]
Fetched 358 kB in 4s (77.5 kB/s)      
Selecting previously unselected package wireguard-dkms.
(Reading database ... 553444 files and directories currently installed.)
Preparing to unpack .../wireguard-dkms_0.0.20200215-0ppa1~16.04_all.deb ...
Unpacking wireguard-dkms (0.0.20200215-0ppa1~16.04) ...
Selecting previously unselected package wireguard-tools.
Preparing to unpack .../wireguard-tools_1.0.20200206-0ppa1~16.04_amd64.deb ...
Unpacking wireguard-tools (1.0.20200206-0ppa1~16.04) ...
Selecting previously unselected package wireguard.
Preparing to unpack .../wireguard_1.0.20200206-0ppa1~16.04_all.deb ...
Unpacking wireguard (1.0.20200206-0ppa1~16.04) ...
Processing triggers for man-db (2.7.5-1) ...
Setting up wireguard-dkms (0.0.20200215-0ppa1~16.04) ...
Loading new wireguard-0.0.20200215 DKMS files...
First Installation: checking all kernels...
Building for 4.4.0-173-generic and 4.9.12-040912-generic
Building initial module for 4.4.0-173-generic
Secure Boot not enabled on this system.
Done.

wireguard:
Running module version sanity check.
 - Original module
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/4.4.0-173-generic/updates/dkms/

depmod....

DKMS: install completed.
Setting up wireguard-tools (1.0.20200206-0ppa1~16.04) ...
Setting up wireguard (1.0.20200206-0ppa1~16.04) ...

2. The following problem troubleshooting steps (lsmod kernel) are from: https://www.kevin-messer.net/how-to-setup-a-vpn-on-ubuntu-19-10-using-wireguard/

As Wireguard is a kernel module, soon to be mainlined (included inside the Linux kernel), you will need to check if it’s enabled and enable it if it’s not.

To check if you will have to do so:

$ lsmod | grep wireguard

If it’s ok, you will get something like this:

root@starlite:~# lsmod | grep wireguard
wireguard             208896  0
ip6_udp_tunnel         16384  1 wireguard
udp_tunnel             16384  1 wireguard

If you get nothing, you will have to enable it:

$ modprobe wireguard
modprobe: ERROR: could not insert 'wireguard': Operation not permitted

If you encountered error, run in verbose mode to check log:

root@starlite:~$ modprobe -vvv wireguard                                                                    
modprobe: INFO: ../libkmod/libkmod.c:364 kmod_set_log_fn() custom logging function 0x559bfb4c2750 registered                                                       
modprobe: DEBUG: ../libkmod/libkmod-index.c:755 index_mm_open() file=/lib/modules/4.15.0-1057/modules.dep.bin                 
modprobe: DEBUG: ../libkmod/libkmod-index.c:755 index_mm_open() file=/lib/modules/4.15.0-1057/modules.alias.bin                    
modprobe: DEBUG: ../libkmod/libkmod-index.c:755 index_mm_open() file=/lib/modules/4.15.0-1057/modules.symbols.bin              
modprobe: DEBUG: ../libkmod/libkmod-index.c:755 index_mm_open() file=/lib/modules/4.15.0-1057/modules.builtin.bin                     
modprobe: DEBUG: ../libkmod/libkmod-module.c:556 kmod_module_new_from_lookup() input alias=wireguard, normalized=wireguard             
modprobe: DEBUG: ../libkmod/libkmod-module.c:562 kmod_module_new_from_lookup() lookup modules.dep wireguard                                
modprobe: DEBUG: ../libkmod/libkmod.c:574 kmod_search_moddep() use mmaped index 'modules.dep' modname=wireguard                 
modprobe: DEBUG: ../libkmod/libkmod.c:402 kmod_pool_get_module() get module name='wireguard' found=(nil)                         
modprobe: DEBUG: ../libkmod/libkmod.c:410 kmod_pool_add_module() add 0x559bfd41c050 key='wireguard'                            
modprobe: DEBUG: ../libkmod/libkmod.c:402 kmod_pool_get_module() get module name='ip6_udp_tunnel' found=(nil)                                       
modprobe: DEBUG: ../libkmod/libkmod.c:402 kmod_pool_get_module() get module name='ip6_udp_tunnel' found=(nil)                                            
modprobe: DEBUG: ../libkmod/libkmod.c:410 kmod_pool_add_module() add 0x559bfd41c180 key='ip6_udp_tunnel'                                       
modprobe: DEBUG: ../libkmod/libkmod-module.c:196 kmod_module_parse_depline() add dep: /lib/modules/4.15.0-1057/kernel/net/ipv6/ip6_udp_tunnel.ko  
modprobe: DEBUG: ../libkmod/libkmod.c:402 kmod_pool_get_module() get module name='udp_tunnel' found=(nil)                                             
modprobe: DEBUG: ../libkmod/libkmod.c:402 kmod_pool_get_module() get module name='udp_tunnel' found=(nil)
modprobe: DEBUG: ../libkmod/libkmod.c:410 kmod_pool_add_module() add 0x559bfd41c2b0 key='udp_tunnel'                                                  
modprobe: DEBUG: ../libkmod/libkmod-module.c:196 kmod_module_parse_depline() add dep: /lib/modules/4.15.0-1057/kernel/net/ipv4/udp_tunnel.ko
modprobe: DEBUG: ../libkmod/libkmod-module.c:202 kmod_module_parse_depline() 2 dependencies for wireguard                                                
modprobe: DEBUG: ../libkmod/libkmod-module.c:583 kmod_module_new_from_lookup() lookup wireguard=0, list=0x559bfd41c160                         
modprobe: DEBUG: ../libkmod/libkmod.c:501 lookup_builtin_file() use mmaped index 'modules.builtin' modname=wireguard               
modprobe: DEBUG: ../libkmod/libkmod-module.c:1750 kmod_module_get_initstate() could not open '/sys/module/wireguard/initstate': No such file or directory
modprobe: DEBUG: ../libkmod/libkmod-module.c:1760 kmod_module_get_initstate() could not open '/sys/module/wireguard': No such file or directory
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=snd_pcsp mod->name=udp_tunnel mod->alias=(null)        
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=snd_usb_audio mod->name=udp_tunnel mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=cx88_alsa mod->name=udp_tunnel mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=snd_atiixp_modem mod->name=udp_tunnel mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=snd_intel8x0m mod->name=udp_tunnel mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=snd_via82xx_modem mod->name=udp_tunnel mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=md_mod mod->name=udp_tunnel mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=bonding mod->name=udp_tunnel mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=dummy mod->name=udp_tunnel mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=nvme_core mod->name=udp_tunnel mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod.c:501 lookup_builtin_file() use mmaped index 'modules.builtin' modname=udp_tunnel
modprobe: DEBUG: ../libkmod/libkmod-module.c:1750 kmod_module_get_initstate() could not open '/sys/module/udp_tunnel/initstate': No such file or directory
modprobe: DEBUG: ../libkmod/libkmod-module.c:1760 kmod_module_get_initstate() could not open '/sys/module/udp_tunnel': No such file or directory
modprobe: DEBUG: ../libkmod/libkmod-module.c:744 kmod_module_get_path() name='udp_tunnel' path='/lib/modules/4.15.0-1057/kernel/net/ipv4/udp_tunnel.ko'
modprobe: DEBUG: ../libkmod/libkmod-module.c:744 kmod_module_get_path() name='udp_tunnel' path='/lib/modules/4.15.0-1057/kernel/net/ipv4/udp_tunnel.ko'
insmod /lib/modules/4.15.0-1057/kernel/net/ipv4/udp_tunnel.ko
modprobe: DEBUG: ../libkmod/libkmod-module.c:744 kmod_module_get_path() name='udp_tunnel' path='/lib/modules/4.15.0-1057/kernel/net/ipv4/udp_tunnel.ko'
modprobe: INFO: ../libkmod/libkmod-module.c:886 kmod_module_insert_module() Failed to insert module '/lib/modules/4.15.0-1057/kernel/net/ipv4/udp_tunnel.ko': Operation not permitted
modprobe: ERROR: could not insert 'wireguard': Operation not permitted
modprobe: DEBUG: ../libkmod/libkmod-module.c:468 kmod_module_unref() kmod_module 0x559bfd41c050 released
modprobe: DEBUG: ../libkmod/libkmod.c:418 kmod_pool_del_module() del 0x559bfd41c050 key='wireguard'
modprobe: DEBUG: ../libkmod/libkmod-module.c:468 kmod_module_unref() kmod_module 0x559bfd41c2b0 released
modprobe: DEBUG: ../libkmod/libkmod.c:418 kmod_pool_del_module() del 0x559bfd41c2b0 key='udp_tunnel'
modprobe: DEBUG: ../libkmod/libkmod-module.c:468 kmod_module_unref() kmod_module 0x559bfd41c180 released
modprobe: DEBUG: ../libkmod/libkmod.c:418 kmod_pool_del_module() del 0x559bfd41c180 key='ip6_udp_tunnel'
modprobe: INFO: ../libkmod/libkmod.c:331 kmod_unref() context 0x559bfd41b490 released

Retry running command with sudo:

ubuntu@starlite:~$ sudo modprobe -vvv
modprobe: INFO: ../libkmod/libkmod.c:364 kmod_set_log_fn() custom logging function 0x55a9e2d4c750 registered                                                       
modprobe: DEBUG: ../libkmod/libkmod-index.c:755 index_mm_open() file=/lib/modules/4.15.0-1057/modules.dep.bin                                                  
modprobe: DEBUG: ../libkmod/libkmod-index.c:755 index_mm_open() file=/lib/modules/4.15.0-1057/modules.alias.bin
modprobe: DEBUG: ../libkmod/libkmod-index.c:755 index_mm_open() file=/lib/modules/4.15.0-1057/modules.symbols.bin                                              
modprobe: DEBUG: ../libkmod/libkmod-index.c:755 index_mm_open() file=/lib/modules/4.15.0-1057/modules.builtin.bin             
modprobe: DEBUG: ../libkmod/libkmod-module.c:556 kmod_module_new_from_lookup() input alias=wireguard, normalized=wireguard             
modprobe: DEBUG: ../libkmod/libkmod-module.c:562 kmod_module_new_from_lookup() lookup modules.dep wireguard                        
modprobe: DEBUG: ../libkmod/libkmod.c:574 kmod_search_moddep() use mmaped index 'modules.dep' modname=wireguard                           
modprobe: DEBUG: ../libkmod/libkmod.c:402 kmod_pool_get_module() get module name='wireguard' found=(nil)                               
modprobe: DEBUG: ../libkmod/libkmod.c:410 kmod_pool_add_module() add 0x55a9e48c1010 key='wireguard'                                        
modprobe: DEBUG: ../libkmod/libkmod.c:402 kmod_pool_get_module() get module name='ip6_udp_tunnel' found=(nil)                   
modprobe: DEBUG: ../libkmod/libkmod.c:402 kmod_pool_get_module() get module name='ip6_udp_tunnel' found=(nil)                    
modprobe: DEBUG: ../libkmod/libkmod.c:410 kmod_pool_add_module() add 0x55a9e48c1140 key='ip6_udp_tunnel'                       
modprobe: DEBUG: ../libkmod/libkmod-module.c:196 kmod_module_parse_depline() add dep: /lib/modules/4.15.0-1057/kernel/net/ipv6/ip6_udp_tunnel.ko
modprobe: DEBUG: ../libkmod/libkmod.c:402 kmod_pool_get_module() get module name='udp_tunnel' found=(nil)                                                
modprobe: DEBUG: ../libkmod/libkmod.c:402 kmod_pool_get_module() get module name='udp_tunnel' found=(nil)                                      
modprobe: DEBUG: ../libkmod/libkmod.c:410 kmod_pool_add_module() add 0x55a9e48c1270 key='udp_tunnel'                                                  
modprobe: DEBUG: ../libkmod/libkmod-module.c:196 kmod_module_parse_depline() add dep: /lib/modules/4.15.0-1057/kernel/net/ipv4/udp_tunnel.ko      
modprobe: DEBUG: ../libkmod/libkmod-module.c:202 kmod_module_parse_depline() 2 dependencies for wireguard
modprobe: DEBUG: ../libkmod/libkmod-module.c:583 kmod_module_new_from_lookup() lookup wireguard=0, list=0x55a9e48c1120                                
modprobe: DEBUG: ../libkmod/libkmod.c:501 lookup_builtin_file() use mmaped index 'modules.builtin' modname=wireguard
modprobe: DEBUG: ../libkmod/libkmod-module.c:1750 kmod_module_get_initstate() could not open '/sys/module/wireguard/initstate': No such file or directory
modprobe: DEBUG: ../libkmod/libkmod-module.c:1760 kmod_module_get_initstate() could not open '/sys/module/wireguard': No such file or directory
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=snd_pcsp mod->name=udp_tunnel mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=snd_usb_audio mod->name=udp_tunnel mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=cx88_alsa mod->name=udp_tunnel mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=snd_atiixp_modem mod->name=udp_tunnel mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=snd_intel8x0m mod->name=udp_tunnel mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=snd_via82xx_modem mod->name=udp_tunnel mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=md_mod mod->name=udp_tunnel mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=bonding mod->name=udp_tunnel mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=dummy mod->name=udp_tunnel mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=nvme_core mod->name=udp_tunnel mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod.c:501 lookup_builtin_file() use mmaped index 'modules.builtin' modname=udp_tunnel
modprobe: DEBUG: ../libkmod/libkmod-module.c:1750 kmod_module_get_initstate() could not open '/sys/module/udp_tunnel/initstate': No such file or directory
modprobe: DEBUG: ../libkmod/libkmod-module.c:1760 kmod_module_get_initstate() could not open '/sys/module/udp_tunnel': No such file or directory
modprobe: DEBUG: ../libkmod/libkmod-module.c:744 kmod_module_get_path() name='udp_tunnel' path='/lib/modules/4.15.0-1057/kernel/net/ipv4/udp_tunnel.ko'
modprobe: DEBUG: ../libkmod/libkmod-module.c:744 kmod_module_get_path() name='udp_tunnel' path='/lib/modules/4.15.0-1057/kernel/net/ipv4/udp_tunnel.ko'
insmod /lib/modules/4.15.0-1057/kernel/net/ipv4/udp_tunnel.ko
modprobe: DEBUG: ../libkmod/libkmod-module.c:744 kmod_module_get_path() name='udp_tunnel' path='/lib/modules/4.15.0-1057/kernel/net/ipv4/udp_tunnel.ko'
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=snd_pcsp mod->name=ip6_udp_tunnel mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=snd_usb_audio mod->name=ip6_udp_tunnel mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=cx88_alsa mod->name=ip6_udp_tunnel mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=snd_atiixp_modem mod->name=ip6_udp_tunnel mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=snd_intel8x0m mod->name=ip6_udp_tunnel mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=snd_via82xx_modem mod->name=ip6_udp_tunnel mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=md_mod mod->name=ip6_udp_tunnel mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=bonding mod->name=ip6_udp_tunnel mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=dummy mod->name=ip6_udp_tunnel mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=nvme_core mod->name=ip6_udp_tunnel mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod.c:501 lookup_builtin_file() use mmaped index 'modules.builtin' modname=ip6_udp_tunnel
modprobe: DEBUG: ../libkmod/libkmod-module.c:1750 kmod_module_get_initstate() could not open '/sys/module/ip6_udp_tunnel/initstate': No such file or directory
modprobe: DEBUG: ../libkmod/libkmod-module.c:1760 kmod_module_get_initstate() could not open '/sys/module/ip6_udp_tunnel': No such file or directory
modprobe: DEBUG: ../libkmod/libkmod-module.c:744 kmod_module_get_path() name='ip6_udp_tunnel' path='/lib/modules/4.15.0-1057/kernel/net/ipv6/ip6_udp_tunnel.ko'
modprobe: DEBUG: ../libkmod/libkmod-module.c:744 kmod_module_get_path() name='ip6_udp_tunnel' path='/lib/modules/4.15.0-1057/kernel/net/ipv6/ip6_udp_tunnel.ko'
insmod /lib/modules/4.15.0-1057/kernel/net/ipv6/ip6_udp_tunnel.ko
modprobe: DEBUG: ../libkmod/libkmod-module.c:744 kmod_module_get_path() name='ip6_udp_tunnel' path='/lib/modules/4.15.0-1057/kernel/net/ipv6/ip6_udp_tunnel.ko'
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=snd_pcsp mod->name=wireguard mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=snd_usb_audio mod->name=wireguard mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=cx88_alsa mod->name=wireguard mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=snd_atiixp_modem mod->name=wireguard mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=snd_intel8x0m mod->name=wireguard mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=snd_via82xx_modem mod->name=wireguard mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=md_mod mod->name=wireguard mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=bonding mod->name=wireguard mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=dummy mod->name=wireguard mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1393 kmod_module_get_options() modname=nvme_core mod->name=wireguard mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1750 kmod_module_get_initstate() could not open '/sys/module/wireguard/initstate': No such file or directory
modprobe: DEBUG: ../libkmod/libkmod-module.c:1760 kmod_module_get_initstate() could not open '/sys/module/wireguard': No such file or directory
modprobe: DEBUG: ../libkmod/libkmod-module.c:744 kmod_module_get_path() name='wireguard' path='/lib/modules/4.15.0-1057/updates/dkms/wireguard.ko'
modprobe: DEBUG: ../libkmod/libkmod-module.c:744 kmod_module_get_path() name='wireguard' path='/lib/modules/4.15.0-1057/updates/dkms/wireguard.ko'
insmod /lib/modules/4.15.0-1057/updates/dkms/wireguard.ko
modprobe: DEBUG: ../libkmod/libkmod-module.c:744 kmod_module_get_path() name='wireguard' path='/lib/modules/4.15.0-1057/updates/dkms/wireguard.ko'
modprobe: DEBUG: ../libkmod/libkmod-module.c:468 kmod_module_unref() kmod_module 0x55a9e48c1010 released
modprobe: DEBUG: ../libkmod/libkmod.c:418 kmod_pool_del_module() del 0x55a9e48c1010 key='wireguard'
modprobe: DEBUG: ../libkmod/libkmod-module.c:468 kmod_module_unref() kmod_module 0x55a9e48c1270 released
modprobe: DEBUG: ../libkmod/libkmod.c:418 kmod_pool_del_module() del 0x55a9e48c1270 key='udp_tunnel'
modprobe: DEBUG: ../libkmod/libkmod-module.c:468 kmod_module_unref() kmod_module 0x55a9e48c1140 released
modprobe: DEBUG: ../libkmod/libkmod.c:418 kmod_pool_del_module() del 0x55a9e48c1140 key='ip6_udp_tunnel'
modprobe: INFO: ../libkmod/libkmod.c:331 kmod_unref() context 0x55a9e48c0450 released

Modprobe run looks fine. FINALLY!!!??? Let’s verify.

$ lsmod | grep wireguard
wireguard             221184  0
ip6_udp_tunnel         16384  1 wireguard
udp_tunnel             16384  1 wireguard

Now you’re ready to get to the next step.

Using Ubuntu as a Client with WireGuard#

WireGuard server was provisoned by Trails of Bits’ Algo VPN.

1. Locate the config file and configure WireGuard:

$ cd ~/dev/work/repo/algo

$ sudo install -o root -g root -m 600 configs/{algo_server_pub_ip}/wireguard/zen.conf /etc/wireguard/wg0.conf

2. Start WireGuard client using systemd:

$ sudo systemctl start wg-quick@wg0

3. Check WireGuard client status:

$ sudo systemctl status wg-quick@wg0

4. Verify WireGuard client connection to server:

$ sudo wg

interface: wg0
  public key: **********sIV0RZ*******=
  private key: (hidden)
  listening port: 52180

peer: ***********seEYg3*********=
  preshared key: (hidden)
  endpoint: {server public ip}:port
  allowed ips: 10.x.x.2/32
  latest handshake: 21 seconds ago
  transfer: 8.88 KiB received, 14.36 KiB sent

4. Check for IP leak.

The WireGuard client IP should be the same as the WireGuard server IP, not the public IP assigned by your ISP. Otherwise, your connection is leaking your ISP IP and to some extent, may even be leaking DNS requests.

$ curl http://zx2c4.com/ip

5. Configure WireGuard to auto-restart connection across OS restart:

$ sudo systemctl enable wg-quick@wg0

That’s all! Happy hacking…