Dynamically load jndi.properties when JMeter is running

An approach to having all test configurations in one place is using the jndi.properties file of JMeter.

When maintaining many JMeter test suites, it's convenient to have the jndi.properties placed together with the JMX file, then when you need to run the test file, you copy the associated jndi.properties into the JMeter bin directory. But this becomes a pain when switching to different test suites, as you will have to replace the jndi.properties with the new one.

Here I explain how to load the jndi.properties file from the same path as the JMX file, without copying the file into the JMeter bin directory. Also opening JMeter directly by opening the JMX file.

• Open the jmeter-t.cmd file located in the bin folder of JMeter in a text editor.
• Find the call to the jmeter command, typically at the end of the file and looks like this:
  call "%~dp0"jmeter -j "%~n1.log" -t "%~nx1" %2 %3 %4 %5 %6 %7 %8 %9
• Modify it to add the Juser.classpath parameter to point to the bin folder of the JMeter (%~dp0 variable) and the path of the JMX folder (%~dp1 variable):
  call "%~dp0"jmeter -j "%~n1.log" -t "%~nx1" -Juser.classpath=%~dp0;%~dp1 %2 %3 %4 %5 %6 %7 %8 %9
• Save the changes and close the file.
• Update the associated default application with *.jmx files:
• Right-click any JMX file and choose Properties from the context menu.
• From the General tab, click the Change button to change the application that opens JMX files.
• From the dialogue, navigate to the bin folder of JMeter and select jmeter-t.cmd.

Now you can open any JMX file in JMeter by double-clicking it, supposing the jndi.properties is located in the same folder, it will be loaded with the test.

Adding "Hard Clean" option to Visual Studio when Clean&Rebuild don't work

At some point, while we code, we need to start clean, rebuild our projects, and throw away all old irrelevant artifacts, to ensure that the version we are testing includes our most recent changes.

While Visual Studio (like any other decent IDE) provides such an option to clean the artifacts, and empty the bin and obj folders, I found it not perfect all the time, like changes made in the appsettings.json file, don't always get updated after the rebuild.

I needed a way to run my .NET projects with the confidence that they include my most recent changes, so I don't stare at the screen and wonder if what I see is really what I just changed.

The safest option to ensure that the artifacts are being rebuilt is to manually delete them. So I found a way to automate this. Bear with me...

Deleting the bin and obj folders using a PowerShell script

I don't remember how I got it, did I write it? did I ask ChatGPT to write it... I don't remember, the important thing here is that it's working exactly as I want it, you can run it on any path and be assured that it will delete any nested bin/obj folders, it's a simple script:

Get-ChildItem -Path "." .\ -include bin, obj -Recurse | ForEach-Object ($_) { remove-item $_.fullname -Force -Recurse }

Remember to set the path of the PowerShell terminal to the root path you want to scrape its bin/obj folders. Running it on the default path of the PowerShell window will probably take a hell amount of time to finish to go through all the nested bin/obj folders.

Adding the Script to Visual Studio external tools

One of the least-known features of Visual Studio is the External Tools, it allows you to plugin any external tool directly into Visual Studio, and it is such a perfect place to add the script so that we run it easily and conveniently.

From the Tools menu select> External Tools...

In the Window, click Add to add a new "external tool", fill in the fields with these values:

Title: Hard Clean (up to you)

Command: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe

Arguments: -ExecutionPolicy RemoteSigned -Command "&{ Get-ChildItem -Path "." .\ -include bin, obj -Recurse | ForEach-Object ($_) { remove-item $_.fullname -Force -Recurse }}

Initial directory: $(SolutionDir)

Tick the "Close on exit" option, then click OK to save.

Now, the "Hard Clean" option will appear under the tools menu and it will do the trick> recursively deleting all bin/obj folders under the current solution path.

How to do code reviews correctly

Introduction

Code review is a special event in the life cycle of software development, it's where ownership is transferred from the developer of the pull request to the reviewer.

AI-generated image by Bing mimics a saying that's believed to be Plato's words:

"Let none but geometers enter here" which depicts how important it was for philosophers to know mathematics.

This ownership implies fulfilling special conditions for the successful delivery of the piece of requirement - the ultimate goal of our craft.

For me, and it's a personal preference, the most I like about this phase (from a developer and reviewer perspective) is the transparency of the mind work, seeing the manifestation of ideas, and exchanging them, a practice that rarely takes place in a such a fast-paced world, which I enjoy, and I believe once you find the joy in it, you'll see code reviews differently.

Roles in code review

To understand the importance of the code review process, we should define and understand the roles involved in it.

Machine-side

Machines are at the heart of our industry, whether you view them as just tools or basic players, they are a fundamental part of the picture, and we can't ignore them even in a topic about a humane practice like code review.

Machines simplify this process by mandating specific requirements that are not subject to change between code reviews, these requirements such as running unit tests and ensuring they pass and ensuring coding standards are applied, etc. This can lead to a faster and more effective code review process, focusing on our part, not the machine part.

Human side

Two human roles (no surprise) coexist in the code review process: the developer and the reviewer, each dictates within his ownership boundaries. These boundaries combined complement the collective ownership of the workpiece. In a perfect world these combined ownership boundaries are not owned by anyone, it's rather what defines an organisation.
There is no advantage here, being the developer doesn't put you in a lower rank than a reviewer, and being the reviewer doesn't put you in a higher rank also. Speaking of the ownership language, there is no higher or lower ownership value, a team member transfers the ownership to the next member. How they value this ownership and adhere to it, is the important factor here.

Developer

Ownership for developers implies that they take full responsibility for the code, from meeting the specifications to adhering to the organization’s coding guidelines and best practices.

Since I'm only interested here in the code review process and the interaction between the developer and reviewer, I won't expand on this phase, as it's a different topic. The interaction starts right before you set your working space free from local commits and push them to the remote server for the code review. It starts from this point because a special first code review takes place, where you, the developer, check out your changes locally in the diff viewer, to look at your changes from another angle raising the bar a little bit, doing some criticism to your work as if another eye looking and reviewing your changes. Once you are comfortable with your changes, the second code review takes place.

Reviewer

This is a special role in the code review process. The reviewer is who makes code reviews an effective process or is just a guard to the gate to the target branch. An effective code reviewer can massively influence the team members, establishing a culture of communication, quality and ownership, as all these values are exercised during the code review. It's even more important in async-style communication environments.

What's expected from you as a code reviewer

1. Read the requirements well, so you understand the problem the PR is attempting to solve.
2. For non-trivial PRs, use your preferred IDE to go through the code. Using features IDEs offer can help you understand/ navigate the code better.
3. Be collaborative, and take ownership of the code as it is yours. When suggesting improvements it might be also good to provide the code with your suggested improvements, this will cut short the guesswork and put you both at the heart of the problem.
4. Test the code as you can, you may be more experienced in the domain than the developer, and not all test cases are covered by unit tests. Your colleague will thank you for catching potential bugs before they slip through production.

Here are some points to keep when you code review:

Your tone should be persistent across PRs: develop your neutral language, and use it with all developers with any experience level. This implies you are motivated by objective reasons, not subjective ones.
Be ready for questions: typically, developers receive questions about decisions they have made, but when you are more experienced as a reviewer, your comments can raise questions from developers, be ready for them, and don't just rephrase your comments but rather answer the question in a more detailed manner.
Prefer questions over assumptions: rather than commenting on a decision the developer had taken by making assumptions, ask them and clarify your point in the question. Questions create conversation, assumptions create justification.

When the reviewer is less experienced than the developer

That is an important result when it's adopted that code reviews are an ownership transitioning process, not a seniority-based process. Anyone in the team can do code reviews, including developers at any experience level as long as they take ownership seriously, which is, again, not related to their experience level.

However, to be an effective process for both sides, it's important (as a less experienced developer in the reviewer role) to pick PRs that you can take ownership of, and not be very hard to follow.
Don't be afraid to ask questions, and don't be overwhelmed by how many things you aren't aware of. For new joiners, reviewing PRs is a great way to learn about the domain, the stack and all the new things you find yourself in.

Experienced developers in the leading roles, should make new joiners feel comfortable reviewing PRs, welcoming their suggestions and extending discussions whenever there is an opportunity to do so in the code review. This can help establish an ownership spirit for new joiners.

The interference of roles

There are no strict boundaries between developer and reviewer, actually, in the code review process, and for effective communication, the roles are just rotated, the developer needs to view his changes from a reviewer's point of view, and the reviewer needs to view the changes from a developer point of view as if he would have done it.

Here are some points to consider (as developer and reviewer) in code review:

Don't leave questions open: Trivial as it may seem, you may forget to answer a question or respond to a comment for several reasons, for example, if a private conversation had been carried out between both of you- developer and reviewer, you may think since the point has been cleared in the conversation there is no need to answer the question in the PR comments. But this is not true, it's been cleared for both of you but for a future reader, it's not.

Stay in context: Code review is more than a private dialogue between two developers, it's a contribution to the knowledge base that belongs to the collective intelligence of the organization along with the formal documentation. Moreover, code review is a critical point in the life of the feature, so it makes sense that our conversation helps in this mission by staying within its boundaries.
Keep your conversation in the context of the PR, express your ideas clearly, reference relevant work items as needed, and don't assume others understand what you do.

Be kind and friendly: it's a conversation between two persons, and we want it to flow and to be effective.

Prefer communication over PR, over private conversation: splitting conversation between the PR and external private chat session harms the context of the PR badly, and makes it hard to rely on the PR as a standalone source of information for any future need, but when a single discussion thread grows unexpectedly, consider moving to a private chat session, then add the conclusions you have both agreed on to the PR discussion.

Use text formatting correctly: know the text formatting tools the platform you are using offers to enhance your writing. Most platforms offer a help link beside the text editor for the text formatting shortcuts, for example, code format, links, quotes, etc.

Formatting text makes writing clear, easy to read and follow which in turn results in smoother communication.

For example, one of the important formatting tricks is the combination of link and code format. For example, if I want to refer to the AddAuthentication method API (provided by Microsoft) in a comment, I'd use this format: 

[`API link text`](URL to the API), i.e.:

[`AddAuthentication(IServiceCollection)`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.dependencyinjection.authenticationservicecollectionextensions.addauthentication?view=aspnetcore-8.0#microsoft-extensions-dependencyinjection-authenticationservicecollectionextensions-addauthentication(microsoft-extensions-dependencyinjection-iservicecollection)).

This is the resulting link that will appear in the discussion:

Here, instead of showing a very long and distracting URL (that is itself irrelevant to the context of the discussion), I'm only showing the piece of information that matters, with a hyperlink that optionally offers more information, and is also enclosed in a code format (using the backtick symbol `) to denote that it's an API not just a hyperlink. It's one of the standard formats in almost every software-related platform (the above image was taken from Stack Overflow).

In some other contexts, though, it might be useful not to hide the URL by formatting, when the referenced URL holds important value, or the URL itself is short and descriptive. For example, see both usages of referencing a GitHub issue and PR in a comment:

“There is an open issue regarding a Postgres integration bug in the provider: https://github.com/org/repo/issues/135”

And

“If the issue with Postgres integration was resolved, I think we are safe to work on this issue, can you please pick it up @developer?”

Can you tell the difference between the two occurrences of the GitHub link (issue and PR- both are fake links)?

In short, sense the significant value- the attention direction.

In the first comment, we want to draw attention to an unexpected event, that is, the bug in the 3rd party library, which is causing issues in our service. The link is short enough and tells there is an issue in the library by just looking at the link in the comment, no need to open the link to understand what the commenter means, and it's the ending of the comment, so no distraction.

In the second comment, we were already aware of the issue, the commenter confirmed it was fixed (you can follow the hyperlink if you want to learn more) but the significant matter here is that we need to work on our issue, since nothing is stopping us now.

You may think these are all minor details, but these aesthetic details form and enrich what I defined previously: the context.

Conclusions

We defined the roles and understood the dynamics of code reviews. Aside from our obligation to do code reviews as a non-negotiated step in the software delivery process, code reviews are a great place to build an accumulative experience while focusing on your work. It's nearly impossible to keep track of all the ongoing changes to the system. It's an effective way to catch up with all the updates you are not directly involved in. Spending little time doing code reviews (compared to the time spent in developing the PRs) would give you a comprehensive experience in a fun and interactive way.

---

Credit to Paul Trees

I'm grateful to be working alongside a magnificent tech lead from whom I learn too much, like how to do code reviews correctly.
Lots of lessons I discussed here are my observations of Paul's reviewing process. I remember when I joined the team I was so amazed by his reviews that I asked him gently to do a live code review for one of my PRs to see him in action, I believed he was hiding some unseen magic!
In this article, I wrote about his magic in code reviews.

Complete setup guide of Elasticsearch, Kibana and APM Server with .NET Agent

Observability is a crucial part of any software product, it helps in understanding the internal state of a system by collecting and analyzing metrics, logs, and traces. This information can be used to troubleshoot issues, identify performance bottlenecks, and make improvements to the system.

Observability is especially important for complex systems, such as distributed systems, where it is difficult to track down the root cause of problems.

There exist several tools out there to help monitor your system's health. Assuming you have an instrumented system, how could you visualize this instrumentation information, and get real-time updates on your system performance?

The topic of observability is divergent and can be spanned across many posts.

But in this post, I chose one topic that posed a tough challenge to me, and that is setting up the APM Server of Elasticsearch, given how scattered their documentation is, and the lack of community support, the challenge was real, hopefully for someone trying to fi I'm going to set up the Elastic APM Server, one of the popular tools to analyze metrics.

Let's get straight to it...

Setting up Elasticsearch

First, create an Elasticsearch container:

docker run --name es01 --net elastic -p 9200:9200 -it -m 1GB docker.elastic.co/elasticsearch/elasticsearch:8.10.4

ToFor information, please refer to: 

https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html#_start_a_single_node_cluster

Run this command, and make sure the container es01 is running:

docker ps

For Rancher Desktop Users

If you are using Rancher Desktop instead of Docker Desktop, you might face this problem:

If the container has stopped unexpectedly (please use docker logs es01  to inspect the reason), you may see this error:

node validation exception\n[1] bootstrap checks failed. 
      You must address the points described in the following [1] lines before starting Elasticsearch.\n
      bootstrap check failure [1] of [1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]", "ecs.version"

For information, please see https://www.elastic.co/guide/en/elasticsearch/reference/current/vm-max-map-count.html#vm-max-map-count

To fix, enter Rancher Desktop shell:

rdctl shell

Then run this command:

sysctl -w vm.max_map_count=262144

To exit the shell, run:

exit

Then restart Rancher Desktop (make sure Rancher Desktop GUI is activated):
rdctl shutdown

rdctl start

An introduction to cryptography, blockchain and bitcoin

This article aims to introduce bitcoin and blockchain to beginners with no prior knowledge of the topic.

Bitcoin is a cryptocurrency monetary system based on cryptography, thus, the name "cryptocurrency",
so a basic understanding of some cryptography functions is required. If you are comfortable with any
of the topics discussed below, feel free to jump to another.

The need for cryptography

Since the early days of human civilization, there was a need to secure communications, so only the involved parties can send and receive messages securely. The process of securing communications went through many stages from using simple cipher languages to using complex math functions.



Figure 1: German Lorenz cipher machine, used in World War II to encrypt very-high-level general staff messages (from Wikipedia)

Cryptography

is a set of means used to secure the communications between the involved parties, such that third parties cannot read the messages or alter them. Cryptography uses mathematical functions to convert plaintext to an unreadable ciphertext. It may produce a reversible ciphertext, in a process called encryption, and the process that converts it back to the plaintext is called decryption, or it can produce an irreversible ciphertext (called hash or digest) in a process called hashing.

There are three types of cryptography

Symmetric cryptography

uses a single key for both encryption and decryption, so both sender and recipient have the same key to encrypt and decrypt messages. It's fast but not secure, as the key is shared and anyone can have it. Example of symmetric algorithms: Data Encryption Standard (DES).

The key: is a very long number, the bigger the key size is, the more the algorithm resists brute-force attacks.

Brute force attack: trying all available keys until a key is found, for a key with size 1024 bits, it means trying 21024 keys.

Asymmetric cryptography

uses two keys that are mathematically correlated, one is public and the other one is private, the public key can be extracted from the private key, but not vice versa, of course. The public key is used to encrypt the plaintext, while the private key is used to decrypt the ciphertext. The owner of the key keeps the private key, no one should know it, while the public key is shared with whoever wants to send encrypted messages to the key owner. Asymmetric cryptography is secure but is slow compared with symmetric cryptography.

Hashing

is a one-way standard function that produces an irreversible digest, it’s like the fingerprint in the sense that every input has its hash, two different inputs must not produce the same hash, and you cannot revert the original input from its hash. Some standard hash functions include; SHA256, SHA512, MD5. However, MD5 has been deprecated due to serious vulnerabilities (see: ref:6). One of the hash function vulnerabilities is the collision, which means; two different inputs produce the same hash, and this can happen due to the limited space of the hash algorithm against the unlimited input possibilities, but SHA256 is considered a strong hash algorithm, it produces a 256 bits long hash (or 64 hexadecimal characters), to see how big is this number, see this video: How secure is 256 bit security?


For example: the SHA256 of the word "abc" is:

ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad


Changing any character, produces totally different hash, for example, the SHA256 of "Abc" is:

06d90109c8cce34ec0c776950465421e176f08b831a938b3c6e76cb7bee8790b


You can play around with SHA256 using this utility: https://andersbrownworth.com/blockchain/hash

(it’s a demo utility for great tutorial by Anders Brownworth, see ref:5)

Digital signature

Like paper signatures, digital signature is used as an authentication method. For example; A wants to send B a message, it doesn’t have to be encrypted, the message itself is not confidential, but there should be a method for B to make sure it’s really from A, and no one has altered the message. A sends the message along with his signature of that message, and B can then check if the message was tampered with. In RSA, digital signature combines encryption and hashing. There are two functions:


Sign: hashes the message and then encrypts the hash with the private key, the output is the digital signature.

Verify: decrypts the digital signature with the public key, the result is compared to the hash of the message sent along with the digital signature, if both are identical, then the message is intact, otherwise it should have changed since it was signed.


Figure 2: Digital signature in RSA


Notice: in digital signature using RSA, the public key encrypts the message after hashing while the private key decrypts the signature to get the hash back. This is the opposite of what happens when using the confidentiality service, where the public key encrypts the message and the private key decrypts the ciphertext. Bitcoin uses the ECDSA algorithm for digital signature, which works differently than RSA.

Cryptography services

Cryptography provides three main services:

1. Confidentiality
2. Non-repudiation
3. Integrity

Each one of those is achieved through special kinds of cryptographic functions.


Confidentiality: This service is used when message confidentiality is required, for example sending a message to a WhatsApp contact uses this service to ensure the message is encrypted with the recipient's public key and at his end, the message is decrypted using his private key.

Non-repudiation: is a security service by which evidence is maintained so that the sender and the recipient of data cannot deny having participated in the communication. Individually, they are referred to as the “non-repudiation of origin” and “non-repudiation of receipt.”. In simple terms, you receive a message from someone in a special "format", and you are sure it's from him, think of "signatures". Digital signature offers non-repudiation

Integrity: making sure the message is sent as it is, and no one alters the message without knowing. Hashing offers integrity.

Now that you have a basic knowledge of cryptography, let’s understand how bitcoin works on top of it.

What is money?

Money is a store of value; it's not a value by itself. You pay 100 L.E. for pizza, you are actually exchanging a piece of gold worth 100 L.E. for this pizza. The gold here is also known as a medium of exchange.

So what makes something, like gold, a medium of exchange or “money”?

Properties of money

According to Wikipedia, these are the main properties that make a medium valid for exchange:

Fungibility: its individual units must be capable of mutual substitution (i.e., interchangeability).
Durability: able to withstand repeated use.
Divisibility: divisible to small units.
Portability: easily carried and transported.
Cognizability: its value must be easily identified.
Scarcity: its supply in circulation must be limited.

Once there's a medium with these properties, it can be used to store value, thus, called money.

What is Bitcoin?

First of all, some might get overwhelmed by the popular bitcoin images and think it’s real especially when it’s depicted as real coins. This is not Bitcoin:


Figure 3: Bitcoin

It’s just a design from artists’ imagination, by the end of this article you will realize that there's no rigid representation of Bitcoin, even at the low level.

What’s confusing most for beginners about bitcoin and other digital currencies, is how we can use intangible assets like bitcoin, as money, and if we could use them, why did we think about a new monetary system, what is wrong with the current monetary system? And many other questions that I will try to answer in this article.

Bitcoin uses peer-to-peer technology to operate with no central authority or banks; managing transactions and the issuing of bitcoins is carried out collectively by the network. Bitcoin is open-source; its design is public, nobody owns or controls Bitcoin and everyone can take part.

Peer-to-peer means it’s a collaborative system, more often called a decentralized monetary system, in contrast to a single authority or central bank controlling the whole thing, the transactions are validated through a group of computers, anyone can participate, they are called nodes, they have the same copy of the blockchain (the public database that holds all transactions ever occurred since the launch of the bitcoin in 2009).

Think of its design as the torrent vs Google Drive.

A decentralized digital currency that is run over the internet is not a new idea, the economic Nobel prize laureate Milton Friedman, gave this visionary statement in 1999

The one thing that’s missing, but that will soon be developed, is a reliable e-cash, a method whereby on the Internet you can transfer funds from A to B, without A knowing B or B knowing A.

Different e-cash systems were introduced before Bitcoin but never adopted widely, until this day in 2008, when Satoshi Nakamoto announced his work on Bitcoin:


Figure 4: Satoshi Nakamoto email letter

Nobody knows who Satoshi is, as his identity remains mysterious, but his work on bitcoin and blockchain is considered revolutionary that is changing the world that we know.

Bitcoin is money?

Most of the newcomers to the cryptocurrency topic, including me a few months ago, don’t understand how some tokens can be treated like money that we know, here’s an analogy that may approach the topic to the money we know (it helped my little brother):

Imagine, in a perfect world, all our financial transactions are made through our bank accounts, and everyone has a bank account, literally everyone, even kids, buy candies from a local shop by their credit card, and public transportation (I’m thinking of Egypt) has means to pay through your credit card. In this perfect world (we already are heading to) no need for cash, your company transfers your salary every month to your account, and your company customers pay to your company by transferring to its bank account. The cash becomes useless, and your account balance becomes just a record in the bank system database, no more than that, which gets incremented if your company pays your salary or you when you sell your old laptop online or to your friend, and get decremented when you go to the shopping mall.

At this stage, fiat money disappears and our wealth becomes just a set of records in the bank's databases. This is exactly how blockchain (the cryptocurrencies database/ledger) works, except for some technical details (important details) that make it competent to the current monetary system. And the bitcoin in this example is your balance records.

The blockchain is the backbone of all cryptocurrencies (Bitcoin, Etheruim, etc), and it has applications other than digital currencies, which makes it really revolutionary.

See this video if you want to know the impact of blockchain:

How the blockchain is changing money and business | Don Tapscott

Money properties in the bitcoin

Let’s see if money properties can be applied to bitcoin:

Fungibility: Any bitcoin is interchangeable with another.

Durability: Bitcoins are there as long as the blockchain is maintained, currently it’s being maintained by thousands of nodes (more on that later). It’s even more durable than the fiat money that is subject to physical failures like tearing or damage.

Divisibility: in most cases, if you want to buy bitcoins, you will mostly buy a small fraction of it, the smallest unit of bitcoin is Satoshi (SAT), one bitcoin worth 108 SAT

Portability: it’s digital, it’s even more portable than fiat money.

Cognizability: the blockchain is public and anyone can download it (https://bitcoin.org/en/download).

Scarcity: there is a limited supply of bitcoin, the bitcoin protocol states that only 21 million BTC are available to be mined, 18 million bitcoin (BTC) have already been mined, only 3 million BTC remains to be mined in the upcoming 100 years (though, this is subject to change).
The decentralization nature of Bitcoin

So why the thinking of a different and controversial monetary system instead of the current known system?


Figure 5: Bitcoin decentralization vs fiat money centralization

The main strong point of the cryptocurrencies is that it’s decentralized, that means that there is no centralized authority or bank that controls the transactions and keeps track of them. Instead, it’s peer-to-peer and public that anyone can contribute to its network. The revolution and the growth of this network remind me of the emergence of the internet in the 90s. Also while the centralized system relies on trust (we put our money in banks because we trust them), the decentralized digital currency relies on cryptography.

But after all, how do all this work together; blockchain, nodes, miners, and cryptography, to make money?


That’s what I will try to answer in the next sections.

The Blockchain

As the name says, the blockchain is a chain of blocks, it’s a type of database (ledger) that keeps track of every transaction since the genesis block (first block added by Satoshi in 2009). Every block contains several transactions (can be 1000 transactions, more or less). Every block references the previous block by its hash.


Figure 6: The main components of a block

The life cycle of a transaction

Say, A wants to send B 1 BTC, A first specifies B’s address (his public key), then signs the transaction with his private key (A’s private key) as approval of the ownership of the bitcoins, and broadcasts this transaction waiting for the confirmation.

The nodes verify that A owns 1 BTC, if it’s true, they move the transaction to the mempool (memory pool) where miners can pick up transactions for mining, until then it’s in an unconfirmed state. Mining is the technique used to prove the work done by the miner and gets awarded for.

The Proof of Work

Miners get awarded with bitcoins for the work they do: adding blocks to the blockchain, but bitcoins are very valuable that there should be a nearly valuable effort miners have to do to get them, this effort is done by the CPUs, that try to solve very complex mathematical problems, that require a special kind of computers, and sadly consume a massive amount of power to add the block to the blockchain, thus winning the precious bitcoins (see for example this video of one of the bitcoin miners: Inside The Cryptocurrency Revolution).

The amount of the bitcoins awarded are defined in the bitcoin code and has restricted rules, such that every 210,000 blocks (about 4 years) get halved, it started with 50 BTCs when the bitcoin was launched, and as of February 2021, the award is 6.25 BTCs. The importance of the mining process is that it’s the main and only source of bitcoins, every bitcoin out there originated from a mining award.
The very complex problem

The problem or puzzle that miners need to solve is simple to understand but very complex to solve.

It requires producing a block hash with a specific number of leading zeros. You know that any input has a different hash, but how do miners change the input (the block)? They are not allowed to alter the block data, they don’t own it. Here comes the block field: the nonce (see Figure 6), it’s a number that should be changed so that the hash of the block changes, miners keep changing it until the hash meets the target difficulty, that’s why all blocks’ hashes start with zeros.

Tip: you can have a look at the real blockchain blocks, through some of the blockchain explorers (see ref:26)

Hacking the blockchain

The structure of the blockchain makes it impossible to hack, the Bitcoin network has thousands of nodes that guard the blockchain against bad actors, they all have the same copy of the blockchain, so if someone tried to modify a block on his copy, he also needs to do this for 51% of the nodes.

For example, if B tried to change the transaction “A sends B 1 BTC” to “A sends B 2 BTC”, then the hash of the containing block will also change and the nonce should be re-evaluated to meet the target difficulty, and if this happened (it will consume a lot of power) the next block hash will change because the previous hash has changed, thus re-evaluating the nonce is required, and so on until the most recent block. If the hacker managed to do this, he will also have to update at least 51% of the nodes (about 5,000 blockchain copies). This work is so insanely costly in terms of power consumption that it may not worth it at all, besides it’s not technically practicable.

See ref:5, it’s a very good visualization of the process.
A sends B x BTC?

I’m using this analogy to simplify the transaction details, but it is more complex than that. Bitcoins are expressed in terms of inputs and outputs:


Figure 7: Transactions references

In the above image (from ref:25): A has total inputs (balance) of 100 BTC, they are called Unspent Transaction Outputs (UTXOs), but once they are referenced in another transaction (transaction C) they become spent and cannot be referenced anymore in another transaction. You will notice that any transaction input is another transaction’s output. Also, there’s no notion of balance, but instead “unspent transaction outputs”, what bitcoin wallets do is calculate the total UTXOs to show your balance.
The change

If A needs to send 3 bitcoins to B and he has 2 UTXOs with 2 BTCs each, that’s a total of 4 BTC, what he will do is send the 4 BTC to two addresses, the first is B’s address with 3 BTC, and the second is his own address with 1 BTC to get the change, as it’s only permitted to reference a whole UTXO. If you didn’t specify the address that receives the change, it will be considered transaction fees.

The coinbase

The block must contain at least one transaction, it’s called the coinbase transaction, it’s a special type of transaction that contains the reward and fees sent to the miner’s address.

See figure 6, the coinbase transaction of block 5 sends 7 BTC to M1 (the address of the miner of that block), this 7 BTC is the total of the mining award + the fees, if the award is 6.25 BTC then the fees are 0.75 BTC.

The Fees

You (the one who sends bitcoins) must pay fees so that a miner picks your transaction, and packages it into a block. These fees differ but the more you pay, the more miners will prioritize your transaction, and the faster the transaction gets confirmed. When all bitcoins (21 million) get mined, the miners will only earn fees.

Summary

I started with some of the important concepts of cryptography, bitcoin relies on cryptography to secure transactions.

We use hashing to ensure the integrity of the data.

Bitcoin is a decentralized cryptocurrency, there’s no entity controlling it like banks, it’s kept secure with cryptography and run by people, also named nodes.

Bitcoins users have an asymmetric key-pair: the public key which acts as their address where others can use to send them BTCs, and the private key which acts as their password, they use to sign transactions to others.

Then we saw how bitcoin and money are related and saw that all money properties can be applied to bitcoin.

The blockchain is the ledger that all transactions ever happened live and will be there forever.

The blockchain is nearly impossible to hack, thanks to the collaborative work of the network nodes and cryptography.

Users send bitcoins by referencing previous inputs sent to them, called Unspent Transaction Outputs (UTXOs).

After the transaction being signed it moves to the network where nodes validate it, if verified, the transaction is moved to the memory pool (mempool).

Miners pick transactions from the mempool, pack them into a block then start the mining process, if they find the nonce, they get awarded with freshly mined Bitcoins and the fees.

The higher the fees, the faster the transaction gets picked by miners thus faster to be confirmed.

The coinbase is a special kind of transaction, it contains the miners' award and fees, it’s the first transaction in the block.

References:

The following references work as complementary to this article to fully understand how bitcoin works, I carefully collected them after watching and reading a lot of resources.

Videos:

(1): Bitcoin Fees and Unconfirmed Transactions - Complete Beginner's Guide

(2): Bitcoin Transaction Details - Part 1

(3): Bitcoin Transactions - from "Send" to "Receive"

(4): What is Proof of Stake? - Earn Passive Income with Staking

(5): Blockchain 101 - A Visual Demo

(6): Hashing Algorithms and Security - Computerphile

(7): How secure is 256 bit security?

(8): Bitcoin Q&A: What is the Role of Nodes?

Medium:

(11): How are transactions validated?. The Bitcoin network's ingenuity is… | by Blair Marshall

(12): How does a bitcoin transaction actually work? | by Blair Marshall

(13): What The Heck is UTXO. Peeking into the Bitcoin Transaction… | by Rajarshi Maitra | BitHyve

Others:

(21): Money

(22): investopedia.com What Happens to Bitcoin After All 21 Million Are Mined?

(23): How bitcoin transactions work | How Does Bitcoin Work? | Get Started with

(24): What is the coinbase?

(25): Bitcoin wiki Transaction

(26): Blockchain Explorer - Search the Blockchain | BTC | ETH | BCH

(27): The bitcoin transaction life cycle (image)

When doing composition, forget about what you inherited!

      I'm revisiting the first chapter from Head First Design Patterns, which is my favorite one so far.

It promotes the composition against inheritance through the strategy pattern, and to put it together, here are the main points of the two approaches as I got from the chapter.

Choosing full inheritance (no polymorphism) may be good if you are looking for 

- Code reuse: Code can be written once and shared among subclasses.

but you should think twice before using inheritance for only that purpose because, on the other hand, this can be at a cost, because:

- Every subclass will share the "default" behavior, with no possible customization, classes lose the singularity advantage.

The first solution that comes to mind is employing polymorphism through using interfaces or abstract classes, that fixes the full inheritance issue so:

Behavior can be customized (any subclass can have its own version of the base class/ interface).

But here are the issue with polymorphism:

- Change is costly, where you have to go through all implementations to change the behavior.

- There may be duplications.

So the strategy pattern comes into play if you want to avoid the red points.

The problem is that you don't know if your implementation needs full inheritance or be overridable. because when inheriting implementation from the base class you're stating strongly, that the subclass will never change in a different direction, and when using polymorphism you're stating that the subclass will always be different than the base class, the problem is that; you're always referencing the base class in a way or another: changes are done with respect to the base class.

In the strategy pattern behaviors of subclasses have their own personality that shouldn't be tied to the base class⁽¹⁾, and you compose them into the subclasses and you can change this behavior in runtime.

Let us have the example used in the HFDP book:

you want to implement different types of ducks in the app, each with fly and quack behaviors,

so here's is the IFlyBehavior

public interface IFlyBehavior
{
    void Fly();
}

IQuackBehavior:

public interface IQuackBehavior
{
    void MakeQuack();
}

The Duck abstract class:

    public abstract class Duck
    {
        public void Display()
        {
            Console.WriteLine($"{Name} Duck");
        }

        public IFlyBehavior flyBehavior;
        public IQuackBehavior quackBehavior;
        public abstract string Name { get; }

        public void Fly()
        {
            flyBehavior.Fly();
        }

        public void Quack()
        {
            quackBehavior.MakeQuack();
        }
    }

For every type of duck, there may be different fly/quack behavior, with the strategy pattern, we create the needed behavior class, implementing the IFlyBehavior or IQuackBehavior and attach it to the concrete duck class, for example, the DecoyDuck implementation looks like this:

public class DecoyDuck : Duck
{
    public DecoyDuck()
    {
        flyBehavior = new FlyNoWay(Name);
        quackBehavior = new MuteQuack(Name);
    }

    public override string Name => "Decoy";
}

this fixes the inheritance issues, except for one issue I found later,  which I named the title of this blog post after, do you see it?...

It's the Name property being translated to the behavior class from the base class:

        flyBehavior = new FlyNoWay(Name);
        quackBehavior = new MuteQuack(Name);

with this translation, all the benefits of the composition are put in danger because now the supposedly inheritance-free components are no longer free, they are tied to the base class with this little weight. 

I found this issue when I wanted to change how I attach behaviors, so instead of initializing behaviors in the subclass constructor, I wanted to add them from outside through two public methods: SetFlyBehavior and SetQuackBehavior to change behaviors in runtime. and this means I will need to send the duck name, for example:

var duck = new DecoyDuck();
duck.SetFlyBehavior(new FlyNoWay("Decoy"))

I'm stating Decoy twice as the type and as the string name, I didn't feel OK with this, the name of the duck is loosely coupled with its type, and they need to be maintained together.

The fix is simple; I removed the name from the IFlyBehavior and made it a separate component without being connected to the duck properties. with this change, I can compose any behavior without worrying about the Duck class.

This is all about it, a good lesson on how strategy pattern with composition can be employed.

Detecting collision algorithm

In the previous post, I talked about how I learned to build a basic game.

Take a look at the game here, play it, and come back.

One challenge I had while developing it, was how can I detect the collision between the square and the balls (I didn't give them cute names yet?!).

I held a paper and drew an imaginary frame that captures the square with a group of balls in different positions:

1- A square surrounded by 7 balls

I wanted to understand how I can know if any of these balls are touching the edges of the square.

Clearly, if the ball touches the square, then the distance between the centerline of the circle and the centerline of the square should equal (diameter+side)/2.

So, let's have a variable named touching that holds that distance. The problem was: which centerline should I consider: the horizontal or the vertical?
Consider this frame:

2- Vertical and horizontal centerlines

Each one gives different distances, and only one is a correct indication to a no collision, but using our human intuition is a start to understand how the natural algorithm works.

They are not touching, which means that the distance between the two centerlines should be greater than touching, which means that the natural algorithm picks the horizontal centerlines (the vertical centerlines says there's a collision!). By noticing multiple circles, I noticed a pattern: if the circle intersects with the vertical extension of the square, then we pick the vertical distance (between the horizontal centerlines), and if the circle intersects with the horizontal extension, we pick the horizontal distance:

3- The circle intersects with the vertical extension

Another example, ball number 4: it intersects with the horizontal extension of the square, then we consider the horizontal distance which is equal to touching.

Ok, the tough part is demystified, now we know how the algorithm will work, but how we know with which extension the circle intersects?

It's easy actually: we calculate the distance between the most right x and the most left x (of both the circle and the square), and the same for the y-axis, the one with greater value is the axis with the extension that intersects with the circle (if it's greater than diameter+side then the circle is outside the two square extensions, like circle #7). The code looks like:

right = Math.max(square.x+square.side, circle.x+circle.rad);

left = Math.min(square.x, circle.x-circle.rad);

bottom = Math.max(square.y+square.side, circle.y+circle.rad);

top = Math.min(square.y, circle.y-circle.rad);

if (right - left > down - top) {

//compare with horizontal distance

}

else {

//compare with vertical distance

}

/*These equations assume that the reference point of the square is at its top left corner, and the reference point of the circle is at its center*/

And that ends the collision algorithm. Maybe there's a known algorithm for this, maybe it's simpler, I didn't search, I got excited with the problem and started solving it my self.