Chris Padilla/Blog
My passion project! Posts spanning music, art, software, books, and more. Equal parts journal, sketchbook, mixtape, dev diary, and commonplace book.
- Redis stores data in-memory. RAM is used instead of disc space to store data
- It is a NoSQL Key/Value Store
- Redis has a built in master/replica pattern. Replica servers can defer to changes in master
- Can be stored on a disc at intervals of 5 minutes or an hour
- These are good for backups and disaster recovery
- You would lose an data stored between the last backup and the failure
- Logs every write operation continuously to the disk
- When restarting, it will use the AOF to rebuild the DB
- Slower restarts
- Potential for bugs in the AOF engine.
- Less overhead structurally
- Less complex to write and manage
- Threads in other language are more expensive as far as memory used compared to Go
Stateless Sessions With Cookies
I'm diving into a large research project around authentication, so get ready for many a blog about it!
This week, an approach to handling email/password login.
Authentication
Authentication is simply verifying someone's identity. Different from authorization, which deals with roles and permissions, or if a user can perform certain actions within your application. Authentication is logging someone in, where authorization is verifying they have access to, say, an admin page or editing functionality.
Email and password is the most ubiquitous approach for authentication. And implementing it only takes a few components.
Storage and Encryption
For a custom solution, email and password combinations can be stored on the DB along with a user's profile. When doing this, password encryption is a vital ingredient in the event of a data leak.
bcrypt is a tried and tested solution here. From their documentation, hashing and checking the password are simple function calls:
// encrypt
bcrypt.genSalt(saltRounds, function(err, salt) {
bcrypt.hash(myPlaintextPassword, salt, function(err, hash) {
// Store hash in your password DB.
});
});
// Load hash from your password DB.
bcrypt.compare(myPlaintextPassword, hash, function(err, result) {
// result == true
});
saltRounds
, if that stands out to you, is the number of iterations of random strings included in the hashing process.
Http and Encryption
All fine and well once the password gets here, but what about when it's being sent to the server? HTTP is simply a plain text protocol. Were it to be intercepted by a malicious party, the email and password combo can be used maliciously.
From the client, we can encrypt with the SHA-256 algorithm, and then decode it on the server.
Here's a client example from MDN:
const text =
"An obscure body in the S-K System, your majesty. The inhabitants refer to it as the planet Earth.";
async function digestMessage(message) {
const msgUint8 = new TextEncoder().encode(message); // encode as (utf-8) Uint8Array
const hashBuffer = await crypto.subtle.digest("SHA-256", msgUint8); // hash the message
const hashArray = Array.from(new Uint8Array(hashBuffer)); // convert buffer to byte array
const hashHex = hashArray
.map((b) => b.toString(16).padStart(2, "0"))
.join(""); // convert bytes to hex string
return hashHex;
}
digestMessage(text).then((digestHex) => console.log(digestHex));
And then on a Node Server, the built in Crypto library can decrypt the password.
Sustaining Session
Great! A user is signed in on the homepage. But, once they navigate to another, how do we maintain that logged in state?
I've actually written on two different approaches before: JWT's and Session Storage. Here I'll talk a bit about server sessions and then focus on a twist on the JWT pattern:
A classic approach is to maintain session data on the server. Once a user is authenticated, a cookie is then sent to be stored on the client browser. That cookie comes along for the ride on every request back to the server with no extra overhead (unlike, say, local storage, which would require writing some logic.) With an authentication token stored on the cookie, the server can verify the token and then confirm that it's from the logged in user.
A nice approach for many reasons! If needed, an admin can manually log the user out if there's suspicious activity with an account. Cookies are also a lightweight and easy to implement technology built into the browser.
One drawback is that the session is tied to the specific server. There's added complexity here in a micro service environment. Maintaining that state may also slow the server down with the added overhead.
Another take on this approach is how Ruby on Rails and the package iron-session still makes use of cookies, but with a "stateless" session from the server.
From the Ruby on Rails guide, the idea is that session IDs are replaced with a session hash that can only be decrypted and validated by your server. In this case, it's the client keeping track of their own session, while the server is simply responsible for approving the token. Decrypted, the cookie may contain basic client info:
{user: {id: 100}}
(A note to still avoid PII (personally identifiable information) or storing passwords here!)
This is similar to using JWT's as authentication tokens. A benefit to using a package like iron session here, though, is that the session cookie comes with encrypted data from a non-spec'd algorithm. JWT, however, is a standard. Unless you encrypt it yourself, it's easy for anyone to decrypt your JWT.
Cinnamon Triads
A work in progress! Hoping to get it out before we skip right over to Winter in Texas π
Animal Gestures
Typing β Strong vs Weak and Static vs Dynamic
Two axis of typing, both with specific meanings:
Static vs Dynamic
This dimension answers the question "are types checked at runtime or compile time?"
TypeScript, Go, and C# are languages that assert types at compile time. If there's a type error, your program simply won't compile (or transpile, in TypeScript's case.)
Dynamically typed languages are only asserted at runtime. With JavaScript, for example, you could ship code that will break due to a type mismatch, but may seem fine in the text editor.
In static typed languages like TypeScript, Go, C#, Java, etc., a variable on declaration requires a type. Here's an example in each language:
let name : string = "Chris" // TypeScript
let name = "Chris" // TypeScript infers the string type
var tickets uint = 2 // Go
price := 3 // Go infers type with the shorthand
String city = "Dallas" // Java
string state = "Texas" // C#
In a case like this, I wouldn't be able to change any of the string variables to an int and vice versa. An error will occur at some point letting me know that I've passed an incorrect value to a variable.
So another way of looking at it is "do my variables hold the type or my values?" Though, we'll see exceptions to this in the next section.
In JavaScript, Ruby, and Python, the value maintains the type, not the variable. Here's an example in python:
favorite_pizza = "Cheese"
favorite_pizza = 100
Strong vs Weak
This dimension is a bit more nuanced. In simple terms, this question answers "Can I work around those types?"
Here's a common example of weak typing in JavaScript:
// JavaScript
const a = 1;
const b = "1"
const c = a + b; // "11"
Python is considered strongly typed, but we don't declare our types. We can't, however, do the above JavaScript string and int addition.
In strongly type languages, conversions need to be explicit. You would need to write something like this to add a string and int:
1 + int("1")
So, Python is a dynamic, strongly typed language.
TypeScript is the opposite of Python. While being statically typed, it's still weakly typed, because JavaScript is weakly typed. This still runs without errors:
const a int = 1
const b string = '1'
console.log(a + b) // 11
That makes TypeScript a weak, statically language.
Beethoven - German Dance No. 1
πΊπͺ©
Ah, Ketchup...
An Intro to Redis
I had the pleasure of taking a look at the Redis server at work this week! So here are some notes from my own exploration β
What is Redis?
Redis is a fully fledged database for storing and persisting various types and shapes of data. The name is short for "Remote Dictionary Server"
A few key features:
The in-memory approach to storing data, in addition to its ability to maintain a master/replica patterns makes it a great fit for being the cache server for applications.
Redis can support multiple database forms by expanding on the "Core" format of key-value pairs. Plugins such as RediSearch, RediGraph, RedisJSON, and RedisTimeseries can support other database models like those used by Elastisearch, Graph based such as Neo4J, and MongoDB.
In a microservice environment, Redis can shine as a cache for multiple databases that have varied data models. Latency is often introduced when multiple services rely on multiple connections to different datasources on top of communicating with one another. Using Redis as a cache on request, then updating the cache if the data differs, can keep a group of microservices lean and quick.
How Does Data Persist?
If you opt to use Redis as a primary database, it begs the question: What happens when the server fails?
The safest way is through replicas. Similar to using Redis as a distributed cache, you can use replicas as back ups. But then again, what if ALL of these fail?
There are a couple of ways this is mitigated:
Snapshotting (dump.rdb files)
Append Only File
The best approach is to use both. Redis recommends a mixed approach and goes into great detail on persistence in their documentation.
From here, you can store those persisted files in a cloud environment separate from Redis. The multiple-location approach firms up the reliability of the system.
Setting Values With an Expiration
Most of the juicy details are in how Redis is implemented in your application or cloud environment. The actually interface from the application is pretty simple.
Here's an example of setting a value with an expiration of 6 hours in Python:
import redis
redis_client = redis.Redis(...options)
namespace = "comments"
user = "Chris"
message = "I love Redis!"
target = f'{namespace}:{user}'
r.set(target, value= message, ex=60*60*6)
# Do more work...
user_comment = r.get(target) # "I love Redis!"
After 6 hours, if we were to try and get the value, there would be none since it would be cleared from the cache.
Someday My Prince Will Come - Churchill
π π° π
Reefapalooza!
Sketches from this week!
Went to Reef-a-palooza with my folks this last weekend! Like anime conventions, but for coral and clownfish (and there was a cosplayer!)
Started watching an MST3K classic β Cave Dwellers!
Still doing loads of gestures. This new-fangled digital pen takes some wrangling!
And, lastly, I'm braving my own album art illustration! This guy will be making an appearance:
Goroutines, Structs, and Pointers in Go
A potpourri of Go features I dove into this week!
Structs
Last week I looked at the Maps data type. It works similar to a JS Object, except all values must be the same type.
Structs in Go are a data structure for saving mixed Data Type values. Here's what they look like:
// Type Declaration
type UserData struct {
firstName string
lastName string
email string
numberOfTickets uint
}
// Can then be used in array or slice
var sales = make([]UserData, 0)
var userData = UserData {
firstName: "Chris",
lastName: "Padilla,
email: "hey@chris.com",
numberOfTickets: 2,
}
// accessing values:
booking.firstName
// in a map, you would use bracket syntax
// booking["firstName"]
This is comparable to a lightweight class in languages like Java and C#. And, of course, the type declaration will look familiar to any TypeScript users.
Goroutines
This is what I've been waiting for!
Goroutines are the key ingredient for Go's main benefit: Lightweight and easy to use concurrency.
The strengths of Go over other languages:
Goroutines are an abstraction of an actual OS thread. They're cheaper and lightweight, meaning you can run hundreds of thousands or even millions without affecting the performance of your application.
Java uses OS threads, takes a longer startup time. Additionally, those threads don't have an easy means of communicating with each other. That's made possible through channels in Go, a topic for another day!
Sample Code
The syntax for firing off a concurrent function is pretty simple:
// Do something expensive and Block the thred
sendTickets()
// Now try this non-blocking approach, creating a second goroutine
go sendTickets()
There's a bit of extra work needed to align the separate Goroutine with your main function. If the above code was all that was in our program, the execution would exit before the result from sendTickets()
was reached.
Go includes sync
in it's standard library to help with this:
import (
"sync"
"time"
)
var wg = sync.WaitGroup{}
wg.Add(1)
go sendTickets()
wg.Wait()
func sendTickets() {
// Do something that takes a while.
wg.Done()
}
When we create a Wait Group, we're essentially creating a counter that will keep track of how many concurrent methods have fired. We add to that counter before starting the routine with wg.Add(1)
and then note when we want to hold for those goroutines to end with wg.Wait()
Already, with this, interesting possibilities are now available! Just like in JavaScript, you can set up a Promise.all()
situation on the server to send off multiple requests and wait for them all to return.
Pointers
Pointers are the address in memory of a variable. Pointers are used to reduce memory usage and increase performance.
The main use case is in passing large variables to functions. Just like in JavaScript, when you pass an argument to a function, a copy is made that is then used within that function. If you were to pass the pointer instead here, that would be far less strain on the app's memory
func bookTickets(largeTicketObject TicketObj) string {
// Do stuff with this large object
return "All set!"
}
Also, if we made changes to this object and wanted the side effect of adjusting that object outside the function, they wouldn't take effect. We could make those changes with pointers, though.
To store the pointer in a var, prepend & to the variable name:
x := 5
xPointer = &x
fmt.Println(xPointer)
// Set val through pointer
*xPointer = 64
fmt.Println(i) // 64
Faber - Fantasia Con Spirito
Spooky music month! This one feels very "Cave of Wonders" from Aladdin. π§ββοΈ
Jet Set Radio Vibes
I give one listen to the Jet Set Radio Soundtrack, and now I get some funny 90s hip hop inspired ditties in my recomendations. Not bad!
Been doing lots of gesture drawing digitally to help get used to the pen. It's a surprising jump from traditional to wacom!
Lastly, I'm sorry to say that Anthony Clark bore the horrible news: #Warioctober has arrived.
I regret to inform you that today is the beginning of #Warioctober , where everyone, be they just or wicked, must honor the terrible creature Wario. Here is Classic Wario. May we be forgiven for what we must do. pic.twitter.com/i99DQTjToR
— Anthony Clark (@nedroid) October 1, 2023
I've tried to escape it, but the punishment for skipping out is much more fowl than the ritual itself.
Tourist's Guide to Go
I'm dipping my toes into Go! I became interested after hearing about how accessible it makes multi-threaded logic to coordinate, especially on cloud platforms.
I've been there with JavaScript and Python: it could be handly for processes running in tandem having the ability to communicate. Seems like Go opens this communication channel in a way that's easier to manage than a language such as Java. (All from hearsay, since I've not dabbled in Java just yet.)
I'm excited to dig into all that! But, as one does, first I'm relearning how string templating works in the new language:
Compile Time Errors
When getting ready to run a main.go
file, checks will be made to ensure there aren't any procedural errors. Go checks for unused variables, ensures types match, and that there are no syntax errors. A nice feature for a server side language, keeping the feedback loop tight!
Static Typing
Like C# and Java, Go is statically typed. On declaration, variables need to be assigned a type.
var numberOfCats uint
When assigning, the type can be inferred:
const conferenceTickets = 50
Variables
Constants and mutable variables exist here. Somewhat like JavaScript, the const
keyword works, and var
functions similarly to how it does in JavaScript. No let
option here.
A shorthand for declaring var is with :=
:
validEmail := strings.Contains(email, "@")
This does not work with the const
keyword, though.
Slices and Arrays
Arrays, similar to how they work in lower level languages, must have a set length defined on declaration:
var bookings [50]string
Slices, however, are a datatype built on top of Arrays that allow for flexibility.
var names []string
Both require that all elements be of the same type, hence the string
type following []
.
Maps
Maps in Go are synonymous with Dictionaries in Python and Objects in JavaScript. Assigning properties can be done with the angle bracket syntax:
newUser := make(map[string]string)
newUser["firstName"] = userName
newUser["email"] = email
newUser["tickets"] = strconv.FormatUint(uint64(userTickets), 10)
Maps can be declared with the make
function:
newUser := make(map[string]string)
For Loops
There's only one kind of For loop in Go. Syntactically, it looks similar to Python where you provide a range
from a list to loop through:
for _, value := range bookings {
names := strings.Fields(value)
firstName := names[0]
firstNames = append(firstNames, firstName)
}
Range here is returning an index and a value. Go will normally error out if there are unused variables. So to denote an intentional skip of the index property, the underscore _
will signal to go that we're ignoring this one.
Package
Like C# Namespaces, files in go can be grouped together as a package. You can do so with the package keyword at the top of the file:
package main
import (
"pet-app/cats"
"fmt"
"strconv"
"strings"
)
func main() {...}
Notice "pet-app/cats": here we can import our packages within the same directory with this syntax. "pet-app" is the app name in the generated "go.mod" file, and "cats" is the package name.
Public and Private Functions
By default, functions are scoped to the file. To make it public, use a capital letter in your function name so it may be used when imported:
func BookTickets(name string) string {
// Logic here
return res
}