<ul><li class="ql-align-justify">People tend to care about something making sense more than they care about something being right.</li><li class="ql-align-justify">It’s why we’re bad at forecasting and easily persuaded and why the economy is driven by stories.</li><li class="ql-align-justify">It’s not your fault. The world is complicated and everyone’s trying to fit what’s going on into the context of their own life experiences.</li></ul>Two factors cause the gap between perception and reality.

Making Sense Is Better Persuasion Than Being Right

The most important events – things that are counterintuitive, disruptive, and swayed by emotion more than calculation – tend to share a few traits that make gathering a good sample of data hard:<ul><li>They’re rare.  </li><li>The data is hard to collect.</li><li>More data intuitively feels better.</li></ul>This is a big problem and there is no solution to it other than giving <a href="http://www.collaborativefund.com/blog/surprise/">yourself room for error</a> in not only in your actions but your beliefs.

Factor 1: The Most Important Events Are the Hardest to Sample

History makes clear that the winner isn’t the person who discovers an idea, but the one who sells it most persuasively.Stories are more powerful than statistics because the most believable thing in the world is whatever takes the least amount of effort to contextualize your own life experiences.People go astray when forgetting that storytelling is like exponential fuel, so a great idea told poorly can be a fraction as powerful as a mediocre idea told persuasively.

Factor 2: We Underestimate how Much More Powerful Stories Are than Statistics

People tend to care about something making sense more than they care about something being right. It’s why we’re bad at forecasting and easily persuaded and why the economy is driven by stories. It’s not your fault. The world is complicated and everyone’s trying to fit what’s going on into the context of their own life experiences. Two big things cause the gap between perception and reality. 1. The most important events are the hardest to sample, so extrapolating their lessons can be misleading. There have been two world wars in the last century. But no one would say, “the average world war lasts five years,” or “the average world war kills 60 million people.” No forecaster would say we’re overdue for a world war based on historic trends. A sample size of two can’t tell us what to expect next. The point may be obvious with world wars. But use a slightly more common case and people start drawing all kinds of false conclusions. Studying things with inadequate samples causes more delusion than intentional deceit. That’s because the ease of explaining things based on what’s readily in front of you meets the intellectual cover of using statistics. The most important events – things that are counterintuitive, disruptive, and swayed by emotion more than calculation – tend to share a few traits that make gathering a good sample of data hard: They’re rare. Ask an economist how many recessions they wish they had to study and they’ll say “ten thousand.” But there have only been 11 recessions since we started systematically collecting good data in the 1940s. That is nothing. If we had as much recession data as we wanted we’d likely realize that half of what we think we know about recessions is wrong. The data is hard to collect. Big surveys are expensive to produce. Gathering a classroom of grad students is cheap and easy. So which do you think happens more? And even when you survey people they often either don’t know the answer but give you one anyways (How much red meat did you eat last year?), or they know the answer but won’t tell the truth (Will you vote for this controversial politician?). This is partly why most studies can’t be replicated. More data intuitively feels better but things change over time, so more data can add in past trends that tell you nothing about what to expect in the future. If something is rare, the tendency is to go way back in history to gather as many examples as you can. But the further back you go, the more likely you are to study a world that operated in a different paradigm. We can study the financial crisis of 1873, but the modern banking system lives in a totally different universe than it did then. Daniel Kahneman is one of the best social scientists of our time. Yet he once admitted: Like most research psychologists, I had routinely chosen samples that were too small and had often obtained results that made no sense. I taught statistics and knew how to compute the sample size that would reduce the risk of failure to an acceptable level. But I had never chosen a sample size by computation. Like my colleagues, I had trusted tradition and my intuition in planning my experiments and had never thought seriously about the issue. That is terrifying. Two things happen when we don’t have good samples for important events. One is that we fool ourselves and are surprised when the world works differently than our data suggested it would. The second is that we can easily fool others, because if you look hard enough you can find a flawed set of data that shows whatever you want it to show. This is a big problem and there is no solution to it other than giving yourself room for error in not only in your actions but your beliefs. 2. We underestimate how much more powerful stories are than statistics. A friend once complained that everything Nassim Taleb has written is a rewording of a Philosophy 101. I don’t know if that’s true, but I’m not bothered if it is because Taleb has done a better job explaining concepts than any Philosophy 101 author before him. And history makes clear that the winner isn’t the person who discovers an idea, but the one who sells it most persuasively. Stories are more powerful than statistics because the most believable thing in the world is whatever takes the least amount of effort to contextualize your own life experiences. People go astray when forgetting that storytelling is like exponential fuel, so a great idea told poorly can be a fraction as powerful as an OK idea told persuasively. The CEO who understands everything about their field except the most important thing – rallying employees behind a mission and vision. The financial advisor who aced the CFA exam but can’t maintain clients’ faith during a downturn. The doctor who knows the cure but can’t get patients to stick to a plan. Even contrarians eventually need to be persuasive. James Grant said: “Successful investing is about having people agree with you … later.” A flaw that frustrates many people is not realizing that virtually every job is a sales job, especially as you move up the ladder: Most careers turn into sales jobs when you get senior enough.— Sam Altman (@sama) December 20, 2017 More on this topic: The Full Reset The Theory of Maybes It’s Hard to Predict How You’ll Respond to Risk

Stashed ideas

Making Sense vs. Being Right

It is important to understand the <a href="https://towardsai.net/p/machine-learning/main-types-of-neural-networks-and-its-applications-tutorial-734480d7ec8e">topology </a>of event-driven architecture to deep dive into various sub-patterns of event-driven architecture. This architecture pattern is based on two main <a href="https://towardsai.net/p/machine-learning/main-types-of-neural-networks-and-its-applications-tutorial-734480d7ec8e">topologies </a>: <ul><li>Mediator </li><li>Broker </li> </ul>

Topology of Event-Driven Architecture

The central mediator plays an important role in mediator <a href="https://towardsai.net/p/machine-learning/main-types-of-neural-networks-and-its-applications-tutorial-734480d7ec8e">topology </a>. It is generally used when it is required to orchestrate various steps with an event through a central mediator. It is helpful for events that have multiple events and needs some level of orchestration to process the event.Example: Design GPS Programm in a Vehicle For overall vehicle safety, navigation, and cost estimation plans, GPS data is considered the starting point for designing a proper GPS for a vehicle. They are intended to provide proper business and safety goals.

Mediator Topology

The mediator component acts as an orchestrator for every initial step. It identifies the type of event or type of data and sends it to a specific Ch<a href="https://towardsai.net/p/machine-learning/building-neural-networks-from-scratch-with-python-code-and-math-in-detail-i-536fae5d7bbf">ann </a>el. Several tools are present to handle mediator topology. Example- jBPM, Mule ESB, Apache Camel, Spring Integration, etc.

Mediator

Event Queue is used to carry the event to the event mediator. It is a kind of <a href="https://sponsors.towardsai.net/">enterprise </a>messaging queue system. There are several messaging systems like RabbitMQ, Kafka, etc.

Queue

The event channel is utilized by the event mediator to pass particular events related to each step in the primary event to the event processors. It can be either message topic or message queue.

Channel

It is a component or service application that contains business logic for the execution of business needs on receiving of events. Nowadays these are microservice-based components. It can be a channel queue consumer as well.

Processor

In Broker topology, the message flow is distributed across the event processor on receiving of event. It is different from a mediator because it is not central. It is useful in chain types of processing. supExample: Design Online Food Ordering System Online food ordering is the process of ordering food from a website.<ul><li>User selects restaurant and orders food.</li><li>Pays for the ordered food.</li><li>Restaurant receives notification</li><li>Delivery partner receives notification</li><li>Restaurant handovers food to the delivery partner</li><li>Delivery partner delivers food to the user</li></ul>

Broker Topology

Deep Dive into Event-Driven Architecture

Technical debt refers to a rushed development process or a lack of shared knowledge among team members. However, in many cases, technical debt is inevitable and is part of a normal software development process.The negative impact of technical debt on businesses is massive.Here are the best ways to prevent and manage technical debt that both startups and enterprise companies can implement today.

Defining Technical Debt

Imagine two software engineers have worked on a new feature. They own all knowledge about this specific feature. However, other software engineers in your team don't have this knowledge. When they have to work on this feature or have to do an implementation that makes use of this feature, they won't have the knowledge required to correctly implement a new feature.Therefore, it's essential that you actively share knowledge about features and important codebase changes.

Knowledge-Based Tech Debt

Design debt often occurs in highly competitive markets or startups where speed to market often is the highest priority.When delivering new features quickly, you're not thinking about the structure of features or the architecture of your codebase. It often gets neglected, making it increasingly harder to add new features. And now you have to fix it as a team.Therefore, design debt is closely linked to structuring features and adhering to design patterns.

Design Debt

Code debt is about writing bad code and not fixing this in time. For instance, a developer wants to quickly merge code without writing sufficient tests or adhering to code standards.Many organizations use automation tools such as pre-commit hooks with code linting to verify code quality. When you don’t implement such code checks, bad code can quickly decline the overall quality of your codebase.

Code Debt

A refactoring week allows your team to resolve open bugs, evaluate the current architecture, and prepare the architecture for the upcoming product features. Benefit: A refactoring week gives the needed breathing room for developers to evaluate and reflect on the code before implementing a new set of features. It’s great for solving big pieces of debt.Drawback: The development process slows down while you are doing refactoring and your team is not solving debt continuously.

Three Best Ways to Fight and Prevent Technical Debt: Code and Architecture Refactoring

A retrospective meeting addresses what went well and what didn’t. It’s an open stage to share feedback without placing blame. It would be best if you focused on improvement.Benefit: You can use retrospective meetings to share updates about the code. Engineers can show what they have accomplished.  Drawback: Stakeholders and managers need to be on board with the vision and give engineers time to organise these meetings.

Start Regular Technical Debt Discussions

Tracking technical debt in the editor allows engineers to:<ul><li>Get full visibility on technical debt</li><li>See context for each codebase issue</li><li>Reduce context switching</li><li>Solve technical debt continuously</li></ul>Benefit: Developers already spend most of their time in the editor so it's the best place to track and report technical issues. Implementing a process for managing technical debt will positively influence engineering team morale and customer satisfaction.Drawback: Starting a new habit takes time and effort from team members and it's impossible to do without a tech debt hero.

Start Tracking Technical Debt in Your Editor

Track metrics related to different types of tech debt.<ul><li>Overall code coverage percentage and code coverage per feature</li><li>Number of failed CI/CD builds</li><li>Bug count per week or month</li><li>Feature throughput</li><li>Issues with non-functional requirements</li></ul>

How to Detect a Growing Technical Debt

<ul><li>Startups can easily solve technical debt by implementing automated tools that verify code quality.</li><li>Large enterprises need to use project management tools to understand which features are in development and who’s working on specific areas of the codebase.</li><li>They should use automated code quality tools to ensure overall codebase health.</li></ul>

How to Deal with Tech Debt Differently

Leaving technical debt unaddressed can result in several issues for your organisation, such as higher total cost of ownership, slower time to market, reduced agility because of a poorly designed architecture, and poor security.Make sure to have a process that makes it easy to share knowledge with team members and resolve technical debt continuously.

The Bottom Line

Kill Technical Debt Before it Kills Your Project

DEPENDING ON WHEN you were born, there’s a good chance you’ve spent either several decades online or have never known an offline world. Whatever the case, the internet and its advertising giants know a huge amount about your life.Amazon, Facebook, and Google all have reams of data about you—including your likes and dislikes, health information and social connections—but they’re not the only ones. Countless murky data brokers that you’ve never heard of collect huge quantities of information about you and sell it on.

Your Identity On The Internet

At this stage it’s going to be very difficult to completely delete yourself from the internet, but there are some steps you can take to remove a lot of it. Removing personal information and deleting accounts is a fiddly process, so it’s better to break it down into a few smaller steps and tackle them over time.

Deleting Yourself From The Web

Collecting and selling your data is big business. In 2019 the US state of Vermont passed a law requiring all companies buying and selling third-party personal information to register: In response, more than <a href="https://www.fastcompany.com/90310803/here-are-the-data-brokers-quietly-buying-and-selling-your-personal-information">120 firms logged their details</a>. They included companies building search tools to look up individuals, firms handling location data, and those specializing in your health data. These companies collect everything from your name, address, and date of birth to your social security number, buying habits, and where you went to school and for how long.

Data Brokers

Among the biggest data brokers are Acxiom, Equifax, Experian, Oracle, and Epsilon. Some, but not all, data brokers let people opt out of having their personal information processed—this also depends on where you are in the world. You’ll often have to contact them via email, fill in online forms, and provide extra identification information.Privacy-focused group <a href="https://yourdigitalrights.org/data-brokers">YourDigitalRights has created opt-out forms</a> for 10 of the biggest data brokers to speed up the process of getting your information deleted. It’s probably best to start opting out of the biggest companies first.

Getting Rid Of Data Brokers

You can’t change the way that Google displays its search results, but there are some limited steps you can take to make sure that what’s displayed is up to date and to remove harmful details, such as doxing attempts. If a web page has been updated by its owner but it isn’t reflected in Google’s search results, you can use its tool to remove outdated content. Google will <a href="https://support.google.com/webmasters/answer/7041154">update its search results</a> for pages that no longer exist or are significantly different to the versions it has indexed previously.

Get Google Search Results Updated

Right to be Forgotten is a principle that was established in European courts in 2014 and was incorporated into GDPR in 2018. This allows certain specific information to be removed from search results, <a href="https://support.google.com/legal/answer/10769224?hl=en">including Google</a>, when relevant criteria are met. Generally, if information about you is in the <a href="https://www.farrer.co.uk/news-and-insights/removing-your-information-from-google-search-results-what-is-the-right-to-be-forgotten/">public interest</a> then it will be very difficult to get it removed from search results.

The Right To Be Forgotten

if you really want to minimize your online presence then you need to track down those old Myspace and Tumblr accounts and remove all traces of them. You need a desktop—and a good chunk of time.Start by making a list of all the old accounts you remember using—email addresses and usernames you’ve used can be helpful—and then work through them one by one. For each, you’ll need to sign in or recover the account and navigate through the deletion process.Justdelete.me has a list of links that point to the deletion pages of everything from Gumtree to Vimeo.

Delete Old Online Accounts

You should also search for your name online and combine it with some other pieces of personal data—for instance your email address or where you live—to see what comes up. If you’re diving deep into your online history and attempting to remove old posts on forums or similar services you may have to email web administrators. If the contact details aren’t clear, as might be the case with really old pages, one starting point is to check the web registration details through a <a href="https://who.is/">WHOIS lookup</a>.

Google Your Name

Even if you’re not deleting your online accounts you can still clean up what data you store online. It’s likely your email account contains thousands of old messages (and attachments) dating back years; your Facebook and Twitter accounts might still have posts on them that you’d rather didn’t resurface publicly.

Clean Up Your Digital History

If you’re using Gmail you can bulk-delete old messages by using the search command “older_than:” and adding a time period (1y or 6m, for instance), and then selecting all messages and deleting them.

Those Gmail Messages

Publicly posted data—either photos or text—is obviously far more likely to be found by others. If you’re considering taking the plunge and deleting your current profiles or existing posts, consider downloading and backing up your posts first. Almost all major social media platforms have backup options in their settings.

Those Public Facebook Posts

Twitter doesn’t have any tools to easily delete all your old tweets in bulk, but third-party services do. Both <a href="https://tweetdeleter.com/">Tweet Deleter</a> and <a href="https://tweetdelete.net/">TweetDelete</a> will get rid of your old tweets. If you’re deleting in bulk, both services can be a little glitchy when handling years of data. Forking out TweetDeleter’s monthly $5.99 cost—you can cancel after one month—may be worth it to delete an unlimited number of tweets at once. Keep in mind that by allowing any third-party service access to your online accounts, they may be able to access information stored within them, such as your direct messages.

Those Tweets You Don't Want People To See

In Facebook head to Settings & privacy, Activity log and select the type of activity you want to delete—ranging from posts to photos you’re tagged in. The tool isn’t the most streamlined if you want to delete years of Facebook usage, but as with all efforts to wipe yourself from the internet, you’ll get better results if you spend more time doing it. Alternatively, you can just <a href="https://www.wired.com/story/how-to-delete-your-facebook-account/">delete your Facebook account entirely</a>.

Deleting Facebook

A lot of the ways to remove yourself from the web are time-consuming and involve a lot of paperwork. There may be some instances where you may want to try to speed things up a little or use legal muscle. It may be sensible to seek legal advice and help remove your data from the web if it involves defamatory statements, explicit photographs, and other harmful content.<a href="https://joindeleteme.com/">DeleteMe</a> will try to remove your data from data brokers selling your information, for example. And <a href="https://www.withjumbo.com/">Jumbo</a> can alert you to data breaches and automatically delete new social media posts after a certain number of days.

Getting Serious About Erasing Yourself From The Web

It’s pretty much impossible to keep your data off the internet entirely, but there are some steps you can take going forward. First, consider how much information you want to proactively put online. When you’re signing up for new online accounts, consider whether you need to enter your personal details or whether it would be better to use a <a href="https://www.wired.com/story/avoid-spam-disposable-email-burner-phone-number/">burner account to mask your identity</a>.

Future Protections

Where possible avoid using Big Tech for all your online activities. Pick a <a href="https://www.wired.com/story/privacy-browsers-duckduckgo-ghostery-brave/">web browser</a> and <a href="https://www.wired.co.uk/article/best-google-search-engine-alternative-privacy">search engine</a> that don’t collect your data; use end-to-end-encrypted apps and <a href="https://www.wired.co.uk/article/whatsapp-signal-disappearing-messages">disappearing messages</a> when appropriate; and understand what data WhatsApp, Instagram, Google, <a href="https://www.wired.co.uk/article/amazon-history-data">Amazon</a>, Spotify and others collect about you.

Don't Use Big Tech

You’ll never be able to get a clean slate—but you can significantly downsize your digital footprint.

6 Ways to Delete Yourself From the Internet

Data bias can have notable implications for research and practical applications. For example, in a Facebook scandal, its AI shockingly asked users if they wanted to continue seeing videos about primates after watching a video featuring Black men. Data bias refers to data sets that don't represent the population in study. Models trained on biased data could contain prejudice. That's why AI Researchers and Data Scientists must be vigilant to ensure models don't contain any bias.

AI models trained with biased data

The most common avenues for collecting training data:<ul><li>Paying for data sets</li><li>Using public data sets</li><li>Sourcing open source content</li><li>Using in-person or field-collected data sets</li></ul> A combination of all four will provide the best training data.

Collect data from a variety of sources.

Ensure the data is diverse. Speakers in the audio or video files should possess a range of characteristics, including locations, dialects, genders, sex, race, and nationality. Sourcing such data could be difficult if you only rely on open-source data.

Ensure diverse data

Once you've ensured your initial data set is diverse, your model can still have bias. That's why you should monitor your model's real-world performance. For example, does your model better predict female speech over male speech? If so, retrain with new datasets to overcome any problem areas.

Monitor real-world performance

This article will look at three ways to limit data bias: collecting data from a variety of sources, ensuring data is diverse, and monitoring real world performa

Making Sense vs. Being Right

Deep Dive into Event-Driven Architecture

Kill Technical Debt Before it Kills Your Project

6 Ways to Delete Yourself From the Internet

What Is Data Bias and How to Avoid It

It's time to

READ

LIKE

A PRO!