# Smiling Dev Consulting - Full Content > Complete documentation with full blog posts and project details for deep context. ## Projects A comprehensive list of projects we've worked on, including descriptions, technologies used, and key details. ### Peak Warehouse Solutions **Link**: https://smiling.dev/projects/peak-warehouse-solutions **Description**: A high-performance Astro + Cloudflare Workers website for a premium steel racking distributor, emphasizing reliability and expertise with stunning racking system showcases and optimized customer acquisition. **Technologies**: astro, cloudflare workers, tailwind css, typescript, shadcn styling, svelte, json-ld **Updated**: 3/16/2026 ### The Law of Importance **Link**: https://smiling.dev/projects/law-of-importance **Description**: An intensive, security-first personality assessment platform with secure user authentication, real-time scoring, and group collaboration features. Built on TanStack Start and Supabase with enterprise-grade architecture and zero-trust security implementation. **Technologies**: tanstack start, react, typescript, supabase, postgresql, shadcn ui, tailwind css, row-level security **Updated**: 3/15/2026 ### Smiling Cookbook **Link**: https://smiling.dev/projects/smiling-cookbook **Description**: A personal recipe and food blog website built with Astro static site generation, Cloudflare Workers, and Pages CMS for content management via GitHub markdown and frontmatter documents. Features ad creation and revenue integration. **Technologies**: astro, cloudflare workers, cloudflare pages, pages cms, github, markdown, frontmatter, tailwind css **Updated**: 3/15/2026 ### Blue Logic Water **Link**: https://smiling.dev/projects/blue-logic-water **Description**: A lightning-fast, SEO-optimized website for a premium water filtration company, built with Astro and Cloudflare Pages delivering sub-1.2s load times and modern design aesthetics. **Technologies**: astro, cloudflare workers, tailwind css, typescript, ssr, json-ld, oklch colors **Updated**: 1/30/2026 ### Sorenson Legacy Foundation **Link**: https://smiling.dev/projects/sorenson-legacy-foundation **Description**: Secure website migration and modernization for a non-profit organization. **Technologies**: Wordpress, MySQL, Wix **Updated**: 1/1/2026 ### East Coast Scholars **Link**: https://smiling.dev/projects/east-coast-scholars **Description**: A complete platform rewrite from AWS Amplify to Supabase, modernizing a tutoring management system with improved performance, SQL database architecture, and enhanced scalability. **Technologies**: supabase, react, shadcn ui, tailwind css, supabase functions, hono, postgresql **Updated**: 12/31/2025 ### Tutor the Future **Link**: https://smiling.dev/projects/tutor-the-future **Description**: A real-time tutoring platform with student-tutor matching, live scheduling, and performance tracking. **Technologies**: aws amplify, react, shadcn ui, tailwind css, aws lambda, aws s3 **Updated**: 6/12/2025 ### FFO Officers **Link**: https://smiling.dev/projects/fractional-financial-officers **Description**: Professional website development for a fractional CFO firm targeting high-growth businesses. **Technologies**: Astro, FormSpree, HTML5, Tailwind CSS, React, Cloudflare Workers **Updated**: 5/24/2025 ### AI Chatbot for Enterprise **Link**: https://smiling.dev/projects/enterprise-ai-chatbot **Description**: Enterprise-grade AI chatbot framework for healthcare compliance and internal operations. **Technologies**: LangChain, React, NextJS, FastAPI, MongoDB, Vector Embeddings **Updated**: 1/1/2025 ### Crocker Ventures **Link**: https://smiling.dev/projects/crocker-ventures **Description**: Modern website redesign for a venture capital firm focusing on user experience. **Technologies**: Wordpress, MySQL **Updated**: 10/31/2024 ### Sublime Events **Link**: https://smiling.dev/projects/sublime-events-and-decor **Description**: Digital transformation for an events company with e-commerce and booking system. **Technologies**: Wix, Calendly **Updated**: 8/6/2024 ## Blog Posts Full content of all blog posts with metadata for comprehensive understanding. ## How to migrate DNS from WIX to Cloudflare **URL**: https://smiling.dev/blog/migrate-wix-dns-to-cloudflare **Author**: Bryson M **Published**: 3/20/2026 **Tags**: wix, dns, cloudflare, insider-secret **Description**: Two ways to point your Wix domain to Cloudflare, including the insider secret about contacting Wix support directly and how to avoid the DNSSEC blocker. If you want to point your Wix domain to Cloudflare (especially to run a Cloudflare Worker website), you'll find that Wix doesn't make it easy. But there are two ways to do it. Either you move yourself to an intermediary domain registrar, or you contact Wix support and get them to change the nameservers for you directly. ## Method 1: Transfer Your Domain to a Third-Party Registrar (The Recommended Way) The most commonly recommended approach—and the one you'll find all over [Cloudflare community forums](https://community.cloudflare.com/t/i-need-to-transfer-a-domain-from-wix-to-cloudflare/531010) and [Reddit threads](https://www.reddit.com/r/CloudFlare/comments/1fjbku8/wix_site_with_cloudflare/)—is to transfer your domain from Wix to a third-party registrar like **GoDaddy**, **Namecheap**, or **Porkbun**. Here's why this is the standard recommendation: Wix doesn't allow you to change your nameservers directly through their interface. So if you want Cloudflare to point your domain to your worker website, Cloudflare needs control of those nameservers. By transferring your domain to a third-party registrar, you get complete control and can point your nameservers to Cloudflare whenever you want. **Steps:** 1. Unlock your domain in Wix 2. Get your authorization code from Wix 3. Initiate the transfer at your chosen registrar (GoDaddy, Namecheap, Porkbun, etc.) 4. Complete the transfer process after 5-7 days of waiting. 5. Log into your new registrar and point your nameservers to Cloudflare's nameservers 6. Set up your DNS records in Cloudflare This approach is straightforward and gives you complete control over your domain going forward. This is the default and recommended way to move off Wix. ## Method 2: Have Wix Support Change Your Nameservers (The Workaround) But here's something most people don't know: **Wix support can technically change your nameservers for you.** I discovered this the hard way, and while it's more tedious than transferring, it can work if you don't want to move your domain off Wix. ![DNS settings showing nameserver configuration between Wix and Cloudflare](src/content/images/dns-from-wix-cloudflare.png) ### The Process First, you need to get the exact nameservers from Cloudflare. Log into Cloudflare, add your domain, and it will give you two nameservers that look something like: - `abel.ns.cloudflare.com` - `katrina.ns.cloudflare.com` (The exact names vary based on your setup.) Next, **contact Wix support via live chat**. This is important—start with live chat, not email. Explain clearly: *"I want to change my nameservers to Cloudflare so that I can point my website to Cloudflare. Here are the exact nameservers I need changed."* Be very explicit about what you need. The live agent will likely say they can help but that you need to follow up via email. Don't worry—that's normal. You'll get an email the next day asking you to confirm the changes. In that email, **reiterate exactly what you want**: the two Cloudflare nameservers that need to replace your current ones. After you confirm, you wait again. In my experience, it took about **2 days after that email confirmation** before I received notice that the DNS nameservers were officially updated in Wix's system. So in total, the timeline is roughly: 1. **Day 1**: Live chat with support 2. **Day 2**: Email confirmation request comes in, you reply with confirmation 3. **Day 4**: DNS nameserver update is complete ### The Confusing Part: Your Wix DNS Won't Update Here's something that will mess with your head: **the Wix DNS interface will never show the new nameservers.** It will always display your old records and old nameservers. This is confusing as hell because you'll think nothing changed. ![Wix DNS showing old nameserver records that won't update](src/content/images/wix-does-not-show-other-nameservers.png) But don't worry. The change *is* happening. You can verify it's working by using third-party DNS propagation tools online. You'll start to see the new Cloudflare nameservers show up gradually. ### DNSSEC Was My Blocker Here's where I got stuck: I had **DNSSEC enabled on my Wix domain**, and it was preventing Cloudflare from completing the domain migration. Cloudflare kept failing to detect that the nameserver change had taken effect. ![Warning about different nameservers being configured at the domain registrar](src/content/images/wix-domains-different-nameservers-warning.png) The fix was to **disable DNSSEC in Wix** before initiating the nameserver change. Once I turned that off and waited a few more hours, Cloudflare finally recognized the change and the migration completed successfully. ### Now What? Once Wix support updates your nameservers and Cloudflare completes the migration, you're all set. Your domain now points to Cloudflare, and you can run your Cloudflare Worker website. If you ever decide you want to fully transfer the domain away from Wix later on, you can do that at any time. ## Which Method Should You Choose? If you don't want the hassle of dealing with support tickets and waiting days for DNS propagation, **transfer your domain to a third-party registrar**. It's cleaner, faster, and gives you more control. If you want to keep your domain with Wix for other reasons, **contact Wix support about changing your nameservers**. It's possible, but you'll need patience and you need to make sure DNSSEC isn't getting in your way. --- ## Rust Shined Over Python for My CLI Tool **URL**: https://smiling.dev/blog/rust-shined-over-python-for-my-cli-tool **Author**: Bryson M **Published**: 3/10/2026 **Updated**: 3/10/2026 **Tags**: Python, Rust, ClI Tools, Rewrite **Description**: I rewrote a 12,000 line python CLI in rust in a week. But the real advantage wasn’t speed or performance, it was confidence! I have really been enjoying the new tooling in Python, let me just start out with that. What the team at Astral has been doing with uv, Ruff, and Typo has really pushed the tooling and developer experience. For a lot of things, Python is the right way to go. It's quick and "fast enough." Most developers are proficient in Python, even if they don't know it well. AI tools can rip through it. So why consider Rust? ## Automation Tooling Here is the context. I was making a new tool for a company that centered around the command line. I'm being intentionally vague about the purpose, but it would take data, convert and standardize it, then upload that data in a special format. Also, uploading the data usually required setting up the environment and data in the application via API calls before uploading a new batch of data. So really, it's mainly just some light conversions and then a bunch of HTTP calls to deal with a funky API design. In this context, Python's performance has no significant difference compared to Rust, considering most things are just waiting for the network. In this case, my reasons were reliability and seeking a feeling of robustness. ## Where Python Started to Feel Difficult While tooling, type hints, and linters can help significantly, ultimately a level of complexity begins to emerge throughout the application. Adding validation with Pydantic and assert statements are all good, but it's hard to reason over so many invariants at once. Also, Python's exception handling is fine, but when compared to Rust, it's hard to go back. Result and Option enum error handling in Rust can be verbose and annoying in some cases, but when you're very concerned with every error and trying to recover from most of them, it's fantastic! I don't want to have tons of try-except blocks everywhere. How can Python functions fail, and how will they fail? I don't have any concrete information for most functions about the types of exceptions they might throw just by looking at the code. It requires a lot of testing. Testing was another area where I thought Python would shine. Mocking is easy—you can mock the database, API, configs, and everything else. In the beginning, Rust was a bit trickier to test. For structs and functions that dealt with API or client inputs, it wasn't clear how to mock that. There's mockito, but then reading some Reddit posts, I realized that traits are often the answer. Instead of making a function like `fn get_posts(&client: &MyClient, data: PostUpload) -> Result<>`, it's better to use a trait: ```rust trait HttpGet { fn get(data: serde_json::Value) -> Result<>; } struct MyClient {} impl HttpGet for MyClient { fn get(data: serde_json::Value) -> Result<> { reqwest.get()? } } fn get_posts(client: impl HttpGet, data: serde_json::Value) { client.get(data)? } ``` So why is this different? Well, now I can make a `MockClient` in the tests module and have the mock client return the response I want. It's extremely clear, testable, and I'm not doing runtime import mocking or clobbering. No more Python monkey-patching where I've messed up enough import strings or pytest autouse fixtures that have tripped me up. Testing surprised me. But given the strictness of the type system, plus the in-file test blocks at the bottom of some files, it made testing very helpful and easy to reason about. It allowed me to worry less about having hundreds of tests (I think I had around 800+ in the Python CLI tool) and more about having a few solid tests that help me "lock down" the behavior of each submodule. Integration tests are still important but tend to be less extensive than Python because I'm not trying to stress every edge case that Python left unclear. ## The Results It took me around 10 weeks to build the Python code. Then there were more requests and features to add that took another 2 weeks. Every change required more testing and made me feel unsure if it would work. Total lines of Python code: Including test files: 23,598 Excluding test files: 11,166 It took me a week to rewrite the Python code in Rust. Certainly, having done it first in Python sped me up significantly. I had already learned the quirks of the internal API system we use, and I'd been bitten by the serialization requirements before. I had plenty of test files and checks that I could simply move over. Lines of code in Rust: 20,055 Lines of code excluding Test Blocks: 14,946 ## Other Benefits I Didn't Expect Distributing a Rust binary is much easier than a Python build. I had instructions for users to clone the repository, install `uv`, and then set up the Python environment. I also built a Docker image they could pull, but not every engineer had their artifactory token set up to pull the private Docker image. However, with Rust, I was able to get cross-compilation working. It now compiles to ARM 64 macOS, Windows x86, and Linux. It's fantastic. I recommend users download the binary directly from the repository's artifacts, but I also have a small MUSL Linux Docker image built for those who prefer using Docker. Another nice benefit is that the Rust binary starts up very quickly. I mentioned that performance wasn't a driving factor, but it does feel nice to have a CLI tool start immediately. It feels crisp and tight! --- ## Stop "Melting" Your Data: Why Polars is the Successor Pandas Needs **URL**: https://smiling.dev/blog/stop-melting-your-data-why-polars-is-the-successor-pandas-needs **Author**: Bryson **Published**: 2/11/2026 **Updated**: 2/12/2026 **Tags**: uv, marimo, python, polars **Description**: Something that impressed me this last week is the buzz surrounding a new Python framework called Polars. It has been touted as the successor to Pandas, boasting a better API and significantly faster performance. Plus, it’s written in Rust, which always draws attention nowadays # Polars, Marimo, and the Joy of Personal Finance Something that impressed me this last week is the buzz surrounding a new Python framework called **Polars**. It has been touted as the successor to Pandas, boasting a better API and significantly faster performance. Plus, it’s written in **Rust**, which always draws attention nowadays—though who knows if that "shiny object" syndrome might backfire in the future. Regardless, I’ve been looking for an opportunity to use it, and I finally found a "real-world" itch to scratch. We all love to dive into personal finances with spreadsheets and code, don't we? Well, no—most people don't. If you're my wife, you definitely don't want to get too far into the weeds. She’s perfectly happy with a simple app on her phone that tracks transactions and tells her if we can spend more or less. But I’m a bit more involved. I’ve been tracking our finances at the end of every month for the last four or five years. My process is simple: once a month, I check all our accounts and enter the balances—positive, negative, debit, credit, checking, or loans—into a spreadsheet. It works well, but I’ve always known I could put more effort into creating better visualizations. This was the perfect chance to try out Polars. ### The Modern Python Ecosystem The first thing that struck me was how much the Python ecosystem has improved. If you're using **uv**, the experience of installing packages or dealing with version updates is incredibly fast. The second realization was that I never _really_ knew the Pandas API. I remember it being tricky; you always felt like you were fighting to get back to your mental model of rows and columns. I used Pandas heavily during university because it was the go-to for data processing. After all, if you're given a CSV, why would you ever want to process it by hand? ### Shifting to Polars Polars was a little trickier at first because it leans into **functional programming**. The idea of piping data through transformations is much more prevalent. While I love the concept, looking at the API felt tough for the first hour or two. However, once it clicked, I grew to love it. I especially appreciated the renamed APIs. For example, while the `melt` API from Pandas still exists, Polars gives you a deprecation warning suggesting the `unpivot` function instead. It’s such a thoughtful detail. I’ve always found the `melt` terminology in Pandas a bit abstract—it’s one of those terms you have to "translate" in your head every time you use it. Switching to `unpivot` feels like the API finally caught up to how I actually think about data; I’m not "melting" something into a puddle, I’m literally reversing a pivot to get my months back into a clean, searchable column. ### The Transformation My approach was to read in the CSV, filter out aggregations or stray cells with random text, and extract the raw data. The original spreadsheet was in a "wide" format—months along the X-axis and account names down the Y-axis. Using a mix of filtering and regex parsing, I converted it into a "narrow" format that data processing systems prefer. ```python df = pl.read_csv(csv_file) df = df.filter(pl.col('Type').is_not_null() & pl.col('Account').is_not_null()) df_narrow = df.unpivot( on=date_columns, index=['Type', 'Account'], variable_name='date', value_name='amount' ) df_narrow = ( df_narrow .filter(pl.col("amount").str.strip_chars() != "-") .with_columns( # 1. Clean and cast to Decimal pl.col('amount') .str.replace_all(r'[$\s,\(\)]', '') .cast(pl.Decimal(precision=None, scale=2), strict=False) .fill_null(0), # 2. Parse the date pl.col('date').str.to_date('%m/%d/%Y') ) .with_columns( # 3. Flip the sign for Credit or Loan types pl.when(pl.col("Type").is_in(["Credit", "Loans"])) .then(pl.col("amount") * -1) .otherwise(pl.col("amount")) .alias("amount") ) ) ``` ### Enter Marimo: A Better Notebook The most fun I had was working with **Marimo**, a reactive notebook that serves as a replacement for Jupyter. I’ve always found Jupyter notebooks frustrating to share. When a data scientist sends you a notebook, it often requires specific inputs and breaks on the first run because the cell execution order is messy. Marimo fixes this by: 1. **Outputting a pure Python file:** It uses decorators like `@app.cell` to turn every cell into a function. 2. **Reactivity:** It builds a dependency graph behind the scenes. When you change an input, it automatically reruns only the downstream cells. 3. **UI Elements:** It has built-in sliders, date ranges, and buttons that make it feel like a web app rather than a static document. The result? Beautiful, instantly updated dashboards that are actually presentable to a manager or coworker. ![](src/content/images/Screenshot%20From%202026-02-11%2017-45-03.png) ![](src/content/images/Screenshot%20From%202026-02-11%2017-47-16.png) ![](src/content/images/Screenshot%20From%202026-02-11%2017-46-12.png) | | | | | --- | --- | --- | | **Feature** | **The "Old" Way (Pandas/Jupyter)** | **The "Modern" Way (Polars/Marimo/uv)** | | **Execution** | Eager (calculates every step immediately) | **Lazy** (optimizes the whole plan before running) | | **Speed** | Single-threaded (uses 1 CPU core) | **Multi-threaded** (uses all available cores) | | **Language** | Python / C | **Rust** (high performance & memory safety) | | **Notebook Format** | `.ipynb` (Messy JSON files) | `.py` **(Clean, versionable Python scripts)** | | **State** | Hidden (out-of-order cells can break code) | **Reactive** (updates downstream cells automatically) | | **Package Mgmt** | pip / conda (can be slow/bloated) | **uv** (extremely fast, lightweight, and modern) | So I'm convinced that going forward my go to data stack is polars, uv, and marimo. If I was actually cunching some large data, and not just a pet project for personal finances, I'd start reaching also for DuckDB as a way to reach and store my data in memory or as another alternative to polars. ### Other Links I fonud about Polars from the community: * [https://towardsdatascience.com/modern-dataframes-in-python-a-hands-on-tutorial-with-polars-and-duckdb/](https://towardsdatascience.com/modern-dataframes-in-python-a-hands-on-tutorial-with-polars-and-duckdb/) * [https://pola.rs/posts/case-citizens/](https://pola.rs/posts/case-citizens/) * [https://www.datacamp.com/tutorial/high-performance-data-manipulation-in-python-pandas2-vs-polars](https://www.datacamp.com/tutorial/high-performance-data-manipulation-in-python-pandas2-vs-polars) --- ## The Real Cost of Choosing the Wrong Website Platform **URL**: https://smiling.dev/blog/the-real-cost-of-choosing-the-wrong-website-platform **Author**: Bryson **Published**: 2/6/2026 **Tags**: Web Development, Astro, WordPress, Squarespace, Static Sites, Performance **Description**: Comparing WordPress, Squarespace, and professional Astro websites. While DIY platforms seem cheaper upfront, the long-term costs in performance, flexibility, and opportunity might surprise you. # The Real Cost of Choosing the Wrong Website Platform When starting a new business or launching a website, one of the first decisions you'll face is choosing a platform. The market is flooded with options, each promising to be the best solution. But are you really saving money by going with the cheapest option? Let's dive deep into comparing three popular approaches: WordPress, Squarespace/Wix, and professional static sites built with modern frameworks like Astro. ## The Upfront Cost Illusion It's easy to look at monthly pricing and make a decision based purely on initial numbers: - **Squarespace/Wix**: $16-$40/month for basic plans, $200-$500/year - **WordPress**: Hosting $5-$30/month, plus theme and plugins, $100-$500/year - **Professional Astro Site**: $2,000-$10,000 upfront development, $10-$50/month hosting At first glance, Squarespace seems like the obvious winner. But this narrow view misses the bigger picture entirely. ## Year One: The Honeymoon Phase In the first year, DIY platforms might actually feel like a good deal. You're getting: - A website that exists - Some basic templates to choose from - Built-in hosting and SSL - An editor that sort of works But even in year one, you're starting to notice the cracks: ### Squarespace/Wix Limitations 1. **Performance Issues**: PageSpeed scores consistently in the 30-60 range 2. **Cookie-Cutter Designs**: Your site looks like thousands of others 3. **Slow Editing**: Each change requires navigating their clunky interface 4. **Limited Customization**: Want to add a specific feature? Good luck. 5. **SEO Handicaps**: Limited control over meta tags, structured data, and performance optimization ### WordPress Problems 1. **Plugin Hell**: Need a feature? Add a plugin. Need another feature? Another plugin. Soon you have 30 plugins conflicting with each other 2. **Security Nightmares**: Constant updates, security patches, and potential vulnerabilities 3. **Performance Degradation**: Each plugin adds weight, slowing your site 4. **Maintenance Burden**: Weekly updates, compatibility checks, and debugging ## Years 2-5: The True Cost Emerges This is where the real numbers start to add up. ### Hidden Costs of Squarespace/Wix **Time is Money**: As a business owner, your time is valuable. Consider: - 5-10 hours/month fighting with the editor trying to make simple changes - Unable to implement custom features your business needs - Slow loading times driving away potential customers - Mobile experience that's mediocre at best Let's do the math: If your time is worth $100/hour (a conservative estimate for a business owner), and you spend 7 hours/month on platform frustrations, that's **$700/month or $8,400/year** in hidden costs. **Lost Opportunities**: - Slow site speeds mean higher bounce rates (every 1-second delay can decrease conversions by 7%) - Limited SEO capabilities mean lower search rankings - Generic designs reduce brand credibility - Missing features mean you can't capitalize on business opportunities **The Cookie-Cutter Tax**: When your website looks like everyone else's, you lose credibility. Potential customers can tell you're using a template. They assume your business is small-time or doesn't care about quality. How much is that costing you in lost deals? ### WordPress: The Maintenance Trap WordPress sites require constant attention: - Plugin updates: 2-4 hours/month - Security monitoring: 2-3 hours/month - Performance optimization: 5-10 hours/quarter - Fixing broken functionality after updates: 5-20 hours/year That's **60-100 hours per year** of maintenance. At $100/hour, that's **$6,000-$10,000 annually** in time costs. Plus: - Premium plugins: $200-$1,000/year - Premium themes: $60-$200/year - Security services: $100-$500/year - Hosting upgrades as site slows down: $300-$1,200/year ## The Professional Astro Advantage Modern static site generators like Astro represent a fundamentally different approach. Here's why they win in the long run: ### Performance That Actually Matters - **PageSpeed scores of 95-100**: This isn't just a vanity metric. It means: - Better search engine rankings - Higher conversion rates - Better user experience - Lower bounce rates Google has stated that page speed is a ranking factor. A faster site literally makes you more money through better SEO and conversions. ### True Flexibility With a professionally built Astro site, you get: - **Custom functionality**: Need a specific feature? Build it. No limitations. - **API integrations**: Connect to any service or platform - **Unique designs**: Your brand stands out, not blends in - **Future-proof**: Built on web standards, not proprietary platforms ### The Maintenance Dream Here's the beautiful part about static sites: - **No plugins to update**: No plugin ecosystem to manage - **No database to maintain**: No database vulnerabilities or corruption - **Minimal security concerns**: Static files are inherently more secure - **Automatic scaling**: CDN handles traffic spikes effortlessly - **Maintenance time**: 2-5 hours/year instead of 60-100 hours ### Real Costs Over 5 Years Let's compare the total cost of ownership over 5 years: **Squarespace ($30/month plan)**: - Subscription: $1,800 - Time costs (7 hours/month × $100/hour × 12 months × 5 years): $42,000 - Lost business opportunity (conservative estimate): $10,000-$50,000 - **Total: $53,800-$93,800** **WordPress**: - Hosting: $2,400 (averaging $40/month) - Plugins/Themes: $2,500 - Maintenance time (80 hours/year × $100/hour × 5 years): $40,000 - Security incidents and recovery: $1,000-$5,000 - **Total: $45,900-$49,900** **Professional Astro Site**: - Initial development: $5,000 - Hosting (CDN): $1,800 (averaging $30/month) - Maintenance time (3 hours/year × $100/hour × 5 years): $1,500 - Minor updates/additions: $2,000 - **Total: $10,300** ## But What If You're On A Tight Budget? This is a legitimate concern. If you're bootstrapping and genuinely cannot afford professional development, here's my honest advice: 1. **Start with Squarespace/Wix if you must**, but: - Plan your migration strategy from day one - Keep content simple and portable - Don't customize too heavily - Set a revenue milestone for upgrading 2. **Avoid WordPress unless you're technical**: The maintenance burden is real and often underestimated 3. **Save for the right solution**: A professional static site is an investment that pays dividends 4. **Consider it a business priority**: Would you rent a storefront in a run-down mall to save money? Your website is often your primary storefront. ## The Reality Check Yes, you can spend less money upfront with Squarespace or Wix. And if you personally enjoy dealing with their limitations, slow editing interfaces, and mediocre performance, you can save some money. But let's be honest about what you're getting: - **Slower sites**: Squarespace and Wix are consistently the slowest platforms in performance benchmarks - **Limited control**: You're at the mercy of their platform decisions - **Generic appearance**: Templates can only be customized so far - **Missing features**: Many business needs simply can't be implemented - **Hidden time costs**: The personal time sink of fighting the platform ## The Professional Developer Advantage When you work with a professional developer to build an Astro site, you get: 1. **Expert guidance**: They know what works and what doesn't 2. **Custom solutions**: Built exactly for your business needs 3. **Performance optimization**: They know how to make sites fast 4. **Clean code**: Easy to maintain and extend 5. **Knowledge transfer**: They can teach you or your team to manage content 6. **Long-term support**: A relationship, not just a transaction ## Conclusion: Choose Wisely The cheapest option is rarely the best option. The real cost of a website isn't just the monthly subscription—it's the time you spend managing it, the opportunities you miss because of limitations, and the customers you lose due to poor performance. **Choose Squarespace/Wix if**: - You have absolute zero budget - You're okay with a mediocre online presence - You enjoy spending hours fighting with a limited platform - Your business doesn't depend on your website **Choose WordPress if**: - You're technical and enjoy maintenance - You're willing to invest significant time in upkeep - You understand the security implications - You want thousands of plugin options (and their conflicts) **Choose a professional Astro site if**: - You value your time - Your business is serious about online presence - You want the best performance possible - You understand the long-term value of quality - You want to focus on your business, not your website platform The web has evolved. Static site generators like Astro represent the modern approach to web development—combining the best of static site performance with the flexibility of dynamic functionality when needed. They're fast, secure, maintainable, and scalable. Your website is often your first impression. Make it count. --- *Smiling Dev Consulting specializes in building high-performance Astro sites that put your business first. If you're tired of fighting with your platform or ready to upgrade from a DIY solution, [let's talk](/contact).* --- ## Why I Ditched My Java Backend for Hono and AI Interns. **URL**: https://smiling.dev/blog/why-i-ditched-my-java-backend-for-hono-and-ai-interns **Author**: Bryson **Published**: 1/21/2026 **Tags**: AI Agents, Spring Boot, Hono, Serverless, AWS Lambda, Backend Development, Rewrite **Description**: A practical look at using AI agents to rewrite a Java Spring Boot backend service into a TypeScript Hono serverless application, achieving better performance with less maintenance overhead. I have started to get more into the world of AI Agents as tasks. Well, I think nearly everyone has. It feels like in some ways the best tool I have right now but also the biggest crutch. This is only **exacerbated** by family life with kids under 5 because they are keenly aware of where I am in the house. I lean on using AI agents often now because I can have a few ideas on how to get something done, walk over to my computer and tell my AI interns how to get it done. The more specific the better. Then I tell them to go to work while I mediate fights between siblings over toys without resorting to screentime. In some ways, it is the best **thing** I can have as a busy dad because it allows **me** to focus on just the high-level problem solving while leaving the details to something else. I imagine I'm not the only one with **these** kinds of work/family commitments that can benefit from an always-on agent. > Sidenote: I really enjoy the process of writing as a way to learn and solidify my own experiments. For that reason, I don't use LLMs to write articles for me. ## Why rewrite? We have a Java service that has been running now for a few months and it does its job. It runs okay enough, but if I'm being honest, I hate working with Java. I can appreciate how widespread and important Java is **as** a language and for infrastructure code, but honestly, it grinds my gears every time I work with it. I'm not that great with all the details either. So every time **it's** my turn to work with this backend service, I'm already leaning on LLMs to do 90% of the work. What is a Java bean? Why **are there** 4 different `.properties` files? Why does every Java component feel the need to add more **boilerplate** code, more configuration, and less actual code that I can reason about? Instead of seeing `this middleware code is running here`, it's some innate knowledge about JWT middleware configured in some file that I need to know about to be proficient. Why Not! I have a fairly small service that has less than 30 routes. It interacts with only 1 database and blob storage, so it should be pretty easy. ## Old to New Architecture For a backend service like this, **it** is fairly small, isn't going to be used by a large audience (less than 100 req/second), and **maintenance** and cost **are** important since I'll probably be the only engineer to really dig into this—everything else in the future will be me again or an engineer just trying to make a quick fix. I **chose** Hono since I can deploy it on Lambda. I'm putting all the routes together so a single **Lambda** can be **an** entire backend. This makes development faster, easier to reason about, and **lessens** complexity when I need to focus on the **inherent** complexity of the rewrite. If this was a larger or more used system, I would reach for some other architecture. Serverless is often great for developer experience, **bursts** of requests, and cost at low **amounts**. But it can fail on latency, cost at high throughput, and background tasks. Since we are not dealing with a system that requires these features, we can be happy with serverless compute using Lambda. The old architecture was a Spring Boot server running on ECS. This should have a faster response time since the server is always **running**, but as I found later on, I was able to match or beat the latency for the response time with the Lambda server for every route. This is probably due to **an** under-provisioned ECS container (0.5 vCPU) or just bad code that calls databases and other internal systems sequentially instead of parallelizing network calls where possible. That is always something nice with a rewrite: some obvious improvements are sometimes easy to catch. > As a side thought, I've wondered if many projects would be improved if you could spend a few hours quickly making the first revision, then throw it away and spend more time building it again now that you have the concept in mind. Kind of the same idea as building many projects as a method of learning. ## What worked pretty well I've found more and more that when I am opinionated and have **an** idea of how I want the system to appear, things turn out better. Without an opinion guiding the design, the design then defaults usually to the most popular pattern in the model's training corpus. This might **be an** older build system like create-react-app, or running everything with NextJS, which is often not the system I want to use. First, I set up all the **apps**, the build system, any dependency that I want to use, linters, formatters, skeleton routes and pages, server functions, etc. I want to make all the personal decisions **myself** and then let the LLM bang out code. Next, I set up skeleton routes that just returned `TODO` in Hono. My focus was on the [OpenAPI generation](https://hono.dev/examples/hono-openapi) so that I can confirm the system is generating the same OpenAPI schemas as **are** currently defined for the Spring Boot service. This ended up working well because I'm matching the old system right from the get-go based on its contract of what it _should_ do. Once I was happy that the Hono server was generating an OpenAPI spec file that was identical to the old system, then I could start on the implementation of the routes themselves. For each route, I had a quicker model like `Claude Haiku 4.5` run through the Java **code** for each route with a prompt like this: >I am working on rewriting this system into another language. For that reason, I need to document everything that happens with the request of `GET /users`. Everything that happens in the Spring Boot service, including authentication, authorization, JWT **assumptions** about claims and audience, middleware, logging, database interactions, database assumptions, internal API calls, possible error locations and other error handling, and response types need to **be** documented in a **thorough** design document. Then I ran this N number of times for the N routes that existed in the backend service. This ended up taking probably an hour, but it produced some pretty decent documentation. Then I took each design document for each route and gave it to a better model like `Claude Sonnet 4.5` or `Claude Opus 4.5` with the instructions to "recreate this API route given the design document." The first few times, it **made** some strange functions, repetitive helper functions, or calls to systems that were not set up. But after some massaging and tweaking, **it** went smooth. The second half of the routes were prompted **and** made almost effortlessly. The helper functions and **util** classes were already set up, so the **implementation** of the routes at that point **was** quite easy. ## The results I used [hyperfine](https://github.com/sharkdp/hyperfine) to benchmark the results. I could have made **something** more **sophisticated**, but often Linux tools are just what the doctor ordered. Below is the example call to the running backend server with Java on ECS. I was kind of shocked because this service was made by another engineer and the response time of 5 seconds was pretty bad. Most of the time **it** is just doing unoptimized scans on a DynamoDB system or an internal API. This could **absolutely** be better with actual search indexes, but this **system** gets such low traffic that a few seconds by an employee is not a big deal. ```bash hyperfine 'curl https://myservice.acme.com/api/users\?limit\=100\&skip\=0 -H "Authorization: Bearer $BEARER_TOKEN"' Benchmark 1: curl https://myservice.acme.com/api/users\?limit\=100\&skip\=0 -H "Authorization: Bearer $BEARER_TOKEN" Time (mean ± σ): 5.035 s ± 0.573 s [User: 0.018 s, System: 0.013 s] Range (min … max): 4.554 s … 6.284 s 10 runs ``` | Metric | Value | | ------------------ | ------- | | Average | 5.035 s | | Standard Deviation | 0.573s | | Min | 4.554s | | Max | 6.284 s | | Count | 10 | ### Hono results | Metric | Value | | ------------------ | ------- | | Average | 0.719 s | | Standard Deviation | NA | | Min | 0.601s | | Max | 1.099 s | | Count | 17 | So it turned out pretty well! Most of the routes had **a** latency improvement from **1.1x** to 4x. Not bad for a **day's work** on a system that we want to set up and basically forget about for the next year. Many systems in enterprise need to be easy to refactor, **yet** will be forgotten before an engineer picks up a story to add a new feature. The systems are often undocumented or worse, incorrectly documented. Making the system **explicit** with Hono's middleware statements, clear with types, and **running** in a serverless environment is a big win for many low-use big company backend services. Other benefits that I didn't expect was that the CICD pipeline was much faster. While the Java service required building the java into a JAR file, packaging as a docker image, uploading to ECR, deploying with AWS CDK, then waiting for the ECS service to restart to a new task definition, the new lambda service could build the javascript into a `dist/` file in 30 seconds, upload to S3 as a zip file in 10 seconds, and then CDK would complete in 2 minutes because lambda can swap out source code much quicker than starting a whole docker image. Nice win that I didn't plan for. --- ## Defying airflow to Amazon ECS **URL**: https://smiling.dev/blog/airflow-deployment-with-aws-ecs **Author**: Bryson **Published**: 11/17/2025 **Updated**: 11/17/2025 **Tags**: Airflow, Python, Apache, ECS **Description**: This was a big writeup of my five days of struggling to deploy airflow to Amazon ECS. May you not have as many difficulties as I did? Strategies ## Executive Summary **Total Commits Analyzed:** 54 **Problem-Solving Commits:** 47 (87%) **Documentation/Cleanup Commits:** 7 (13%) ### Problem Distribution by Category | Category | Count | Percentage | Top Issues | |----------|-------|------------|------------| | **Airflow-Specific Pitfalls** | 23 | 43% | Configuration, permissions, secrets, service communication | | **AWS/ECS Pitfalls** | 19 | 35% | Secret injection, health checks, resource allocation, spot instances | | **Networking Pitfalls** | 12 | 22% | IPv6, service discovery, ALB configuration, security groups | | **CDK Pitfalls** | 8 | 15% | Context management, synthesizer, stack dependencies | | **Docker/Container Pitfalls** | 15 | 28% | Build performance, user permissions, multi-stage builds | _Note: Commits can appear in multiple categories_ --- ## 1. Airflow-Specific Pitfalls (23 commits) ### Key Learnings - Airflow has complex inter-service dependencies that must be carefully orchestrated - Secret management and encoding are critical for multi-service deployments - DAG deployment strategy significantly impacts architecture complexity - Airflow 3.x has different requirements than 2.x ### Issues Encountered #### 1.1 Configuration & Environment Variables (8 commits) **Problem:** Airflow requires precise configuration across multiple services with shared secrets. | Commit | Issue | Resolution | |--------|-------|------------| | #39 | Removed static airflow.cfg, relied on env vars | Use environment variables exclusively instead of config files | | #40 | ECS injecting "\[object Object\]" for secrets | Fixed secret reference syntax in CDK | | #41 | Secrets still passed incorrectly | Simplified to direct secret ARN references | | #28 | Changed service startup based on local dev | Validated config locally first before ECS deployment | | #4 | Major refactoring of environment variables | Centralized env var management in CDK constructs | | #54 | Removed unneeded default env vars | Clean up reduces complexity and confusion | | #31 | Database connection failing | Proper connection string format with all required params | | #30 | Special characters in DB password broke startup | URL-encode passwords with special characters | **Resolution Strategy:** - ✅ Use environment variables exclusively, avoid static config files - ✅ Validate configuration locally with Docker Compose before ECS - ✅ URL-encode all connection strings and secrets - ✅ Use CDK constructs to centralize environment variable management - ✅ Reference secrets by ARN directly, not through complex wrapper objects #### 1.2 Database & Storage (6 commits) **Problem:** Database connectivity and encoding issues caused repeated failures. | Commit | Issue | Resolution | |--------|-------|------------| | #30 | Encoded chars in password caused ECS failure | URL-encode passwords, avoid special chars in generation | | #31 | DB connection failing | Security groups, connection strings, timeouts | | #36 | Health check failed due to DB connection | Increase grace period, validate connection in startup | | #23 | Bad IPv6 URL when connecting to Postgres | Use proper endpoint format without IPv6 | | #22 | Switched back to PostgreSQL from alternative | Stick with well-supported databases | | #29 | Moved DB migration to API server | Ensures DB ready before other services start | **Resolution Strategy:** - ✅ Generate passwords without special characters OR properly URL-encode them - ✅ Run database migrations in the webserver/API startup, not scheduler - ✅ Use RDS endpoint hostname format, avoid IPv6 references - ✅ Increase health check grace periods for DB-dependent services - ✅ Validate DB connectivity before starting service processes #### 1.3 Service Communication & Architecture (5 commits) **Problem:** Airflow services need to communicate with each other reliably. | Commit | Issue | Resolution | |--------|-------|------------| | #46 | Worker can't reach base URL | Added service discovery and internal ALB | | #47 | Added common JWT secret | Shared JWT for inter-service authentication | | #27 | Can't get auth to work | Proper authentication backend configuration | | #32 | Successfully deployed API and scheduler | Validated basic service communication | | #24 | Split into storage/service stacks | Separate stateful from stateless resources | **Resolution Strategy:** - ✅ Use AWS Cloud Map for service discovery between components - ✅ Configure internal ALB DNS for workers to reach webserver - ✅ Share authentication secrets (JWT, Fernet) across all services - ✅ Separate stateful (DB, cache) from stateless (compute) in stack design - ✅ Deploy webserver/API first, then scheduler, then workers #### 1.4 DAG Deployment & Storage (7 commits) **Problem:** Finding the right way to deploy DAGs to all Airflow services. | Commit | Issue | Resolution | |--------|-------|------------| | #15 | Tried EFS after 'aws' CLI not found in image | EFS for shared storage | | #17 | EFS mount point issues | Fixed mount configuration | | #18 | EFS DNS modification issues | Changed mount attachment method | | #19 | Switched from EFS to git-sync | Git-sync sidecar pattern | | #20 | Added git-sync with SSL certs | Proper SSL for private repos | | #21 | Git-sync needs root permissions | Run as root to write to filesystem | | #43 | Read-only DAG folder caused startup error | Make DAG folder writable | | #49 | Simplified git-sync configuration | Direct git sync approach | | #53 | Fixed git-sync repo configuration | Correct repo URL and credentials | **Resolution Strategy:** - ✅ Use git-sync sidecar pattern instead of EFS (simpler, more reliable) - ✅ Run git-sync as root with proper permissions - ✅ Ensure shared volume is writable by Airflow processes - ✅ Mirror git-sync image to ECR with SSL certificates for private repos - ✅ Configure git-sync to sync to correct path (/opt/airflow/dags) #### 1.5 Component Health & Scaling (6 commits) **Problem:** Getting all Airflow components (webserver, scheduler, worker, triggerer) healthy. | Commit | Issue | Resolution | |--------|-------|------------| | #33 | Triggerer failing health checks | Temporarily disabled to isolate issues | | #34 | Deployed without health checks | Removed to isolate startup from health check issues | | #35 | All but worker running | Webserver, scheduler, triggerer operational | | #44 | Re-enabled triggerer | Fixed config and re-added successfully | | #45 | Worker killed due to high memory | Doubled memory to 2GB | | #51 | Added worker auto-scaling | Scale 1-10 based on CPU/memory | **Resolution Strategy:** - ✅ Disable health checks initially to isolate startup issues - ✅ Deploy services incrementally: webserver → scheduler → triggerer → worker - ✅ Monitor memory usage and increase allocations (workers need 2GB+) - ✅ Implement auto-scaling for workers (1-10 instances) - ✅ Re-enable health checks after confirming services start successfully #### 1.6 Permissions & User Management (3 commits) **Problem:** Container user permissions for filesystem access. | Commit | Issue | Resolution | |--------|-------|------------| | #37 | Trying to fix airflow user with root permissions | User/group permission conflicts | | #21 | Git-sync needs root permissions to write | Run git-sync as root | | #38 | Simplified user management in Docker | Use base image defaults | **Resolution Strategy:** - ✅ Run git-sync sidecar as root (uid 0) for file system writes - ✅ Run Airflow processes as airflow user (default in base image) - ✅ Use shared volumes with appropriate permissions - ✅ Don't override user settings unless necessary --- ## 2. AWS/ECS Pitfalls (19 commits) ### Key Learnings - ECS has specific requirements for secret injection and environment variables - Health checks need careful tuning for startup times - Resource allocation (CPU/memory) requires iteration based on actual usage - Spot instances can significantly reduce costs ### Issues Encountered #### 2.1 Secret Management (5 commits) **Problem:** ECS secret injection has specific syntax requirements. | Commit | Issue | Resolution | |--------|-------|------------| | #40 | ECS injecting "\[object Object\]" literally | Fixed CDK secret reference syntax | | #41 | Secrets still passed incorrectly | Used direct secret ARN references | | #47 | Added common JWT secret | Centralized secret in Secrets Manager | | #52 | Fixed Fernet key generation | Proper secret generation in storage stack | | #30 | Password encoding issues | URL-encode or avoid special characters | **Resolution Strategy:** - ✅ Use `ecs.Secret.fromSecretsManager(secret)` or direct ARN references - ✅ Avoid complex object wrapping in secret definitions - ✅ Store all secrets in Secrets Manager, not environment variables - ✅ Generate secrets in storage stack, reference in service stack - ✅ Use Secrets Manager rotation for production environments #### 2.2 Health Checks & Service Startup (8 commits) **Problem:** ECS health checks failed due to timing and configuration issues. | Commit | Issue | Resolution | |--------|-------|------------| | #42 | ALB considering task failed, need more time | Increased grace period and check intervals | | #36 | Health check failed due to DB connection | Added DB connection validation | | #34 | Deployed without health checks temporarily | Isolated startup from health check issues | | #33 | Triggerer failing health checks | Disabled until config fixed | | #35 | Most services up without health checks | Validated core functionality first | | #15 | Health check failed, probably DB connection | Database initialization timing | | #12 | Added ECS circuit breaker | Automatic rollback on deployment failures | | #54 | Removed unneeded health checks | Cleaned up redundant checks | **Resolution Strategy:** - ✅ Set health check grace period to 300-600 seconds for DB-dependent services - ✅ Use circuit breaker for automatic rollback on failures - ✅ Disable health checks temporarily to isolate startup issues - ✅ Validate database connectivity before enabling health checks - ✅ Use ALB target group health checks for webserver - ✅ Configure startup scripts to wait for dependencies #### 2.3 Resource Allocation (5 commits) **Problem:** Services need adequate CPU and memory resources. | Commit | Issue | Resolution | |--------|-------|------------| | #45 | Worker killed due to high memory usage | Doubled memory to 2GB | | #42 | Bumped to 1 vCPU | Increased from 0.5 to 1 vCPU | | #51 | Added auto-scaling for workers | Scale 1-10 based on metrics | | #45 | Added Spot instances | Cost savings on worker fleet | | #11 | Added temp directory configuration | Proper temp storage allocation | **Resolution Strategy:** - ✅ Start with minimum: webserver/scheduler 1 vCPU/2GB, workers 1 vCPU/2GB - ✅ Monitor CloudWatch metrics for CPU/memory utilization - ✅ Use Spot instances for workers (50-70% cost savings) - ✅ Implement auto-scaling based on CPU (70%) and memory (80%) targets - ✅ Configure temp directories with adequate storage #### 2.4 Container Architecture (4 commits) **Problem:** ARM64 vs AMD64 architecture differences. | Commit | Issue | Resolution | |--------|-------|------------| | #13 | Trying ARM64 for [entrypoint.sh](http://entrypoint.sh) problem | Switched to ARM64 architecture | | #14 | Specific ARM64 tagging | Explicit platform tagging | | #12 | Architecture differences in ECS | Set explicit architecture in task definitions | | #8 | Multi-stage builds with UV | Optimized for target architecture | **Resolution Strategy:** - ✅ Choose ARM64 for better price/performance (Graviton processors) - ✅ Tag Docker images with explicit platform (--platform linux/arm64) - ✅ Set runtimePlatform in ECS task definitions - ✅ Test locally with matching architecture #### 2.5 Availability & Deployment (3 commits) **Problem:** Availability zone and deployment configuration issues. | Commit | Issue | Resolution | |--------|-------|------------| | #6 | Getting error about availability zones | Added AZ mappings to context | | #7 | Some services not starting up | Fixed ECR image references | | #25 | Fixed subnet AZ and naming | Proper subnet selection | **Resolution Strategy:** - ✅ Cache AZ lookups in cdk.context.json to avoid API calls - ✅ Use multiple AZs for high availability - ✅ Ensure subnets span multiple AZs properly - ✅ Use CDK context for consistent AZ selection --- ## 3. Networking Pitfalls (12 commits) ### Key Learnings - Service-to-service communication requires proper security groups and discovery - ALB configuration is critical for external and internal traffic - IPv6 can cause unexpected connection issues - VPC design impacts cost and complexity ### Issues Encountered #### 3.1 Service Discovery & Internal Communication (4 commits) **Problem:** Services couldn't reliably communicate with each other. | Commit | Issue | Resolution | |--------|-------|------------| | #46 | Worker can't reach base URL | Added AWS Cloud Map service discovery | | #46 | Worker to API connectivity | Configured internal ALB DNS | | #46 | Security group rules missing | Added worker-to-webserver connectivity | | #31 | DB connection failing | Security group for RDS access | **Resolution Strategy:** - ✅ Use AWS Cloud Map for service discovery (servicename.namespace) - ✅ Configure internal ALB for worker-to-webserver communication - ✅ Set up security groups to allow inter-service traffic - ✅ Use private subnets for services, public for ALB only - ✅ Document all security group rules for troubleshooting #### 3.2 Load Balancer Configuration (5 commits) **Problem:** ALB health checks and routing configuration. | Commit | Issue | Resolution | |--------|-------|------------| | #42 | ALB considering task failed | Increased timeouts and grace periods | | #2 | Initial ALB setup with webserver | Public ALB for external access | | #46 | Added internal communication path | Internal ALB/service discovery | | #35 | Health check configuration | Proper health check paths | | #54 | Removed unneeded health checks | Simplified configuration | **Resolution Strategy:** - ✅ Use public ALB only for webserver/UI - ✅ Set health check path to /health or /api/v1/health - ✅ Configure deregistration delay to 30 seconds - ✅ Set healthy/unhealthy threshold appropriately (2/3) - ✅ Use longer intervals (30s) for services with slow startup #### 3.3 Database Connectivity (3 commits) **Problem:** RDS connection string format and network access. | Commit | Issue | Resolution | |--------|-------|------------| | #23 | Bad IPv6 URL when connecting to Postgres | Use hostname format without brackets | | #31 | DB connection failing | Security groups and connection params | | #36 | DB connection in health checks | Validate connection before health checks | **Resolution Strategy:** - ✅ Use RDS endpoint hostname directly: `dbinstance.xxxxxx.region.rds.amazonaws.com` - ✅ Avoid IPv6 format with brackets: `[::1]` - ✅ Configure security group ingress on port 5432 from ECS tasks - ✅ Use private subnets for RDS (never public) - ✅ Test connection string format locally first #### 3.4 VPC & Subnet Configuration (3 commits) **Problem:** Subnet and availability zone configuration. | Commit | Issue | Resolution | |--------|-------|------------| | #6 | AZ errors | Fixed AZ configuration and context | | #25 | Subnet AZ issues | Proper subnet selection in correct AZs | | #24 | VPC in storage stack | Centralized VPC in stateful stack | **Resolution Strategy:** - ✅ Create VPC in storage/stateful stack - ✅ Use 2-3 AZs for high availability - ✅ Separate public and private subnets - ✅ Use NAT gateway for private subnet internet access - ✅ Cache AZ lookups in cdk.context.json --- ## 4. CDK Pitfalls (8 commits) ### Key Learnings - Stack organization impacts maintainability and deployment - Context management is crucial for consistent deployments - CDK warnings should be addressed early - Stack dependencies must be explicitly defined ### Issues Encountered #### 4.1 Stack Organization (3 commits) **Problem:** Monolithic stack vs separated concerns. | Commit | Issue | Resolution | |--------|-------|------------| | #24 | Split into storage and service stacks | Separated stateful from stateless | | #4 | Major stack refactoring | Better organization and structure | | #2 | Initial monolithic stack | Started with everything in one stack | **Resolution Strategy:** - ✅ Separate stateful resources (VPC, RDS, Cache) into storage-stack - ✅ Put stateless compute (ECS services) in service-stack - ✅ Use stack outputs and cross-stack references - ✅ Define explicit dependencies between stacks - ✅ Benefits: independent lifecycle, easier testing, faster iteration on services #### 4.2 Context & Configuration Management (3 commits) **Problem:** CDK context and synthesizer warnings. | Commit | Issue | Resolution | |--------|-------|------------| | #5 | Fixing CDK warnings | Added proper synthesizer configuration | | #6 | AZ lookup issues | Added cdk.context.json with AZ mappings | | #6 | Added .env file | Environment-specific configuration | **Resolution Strategy:** - ✅ Use cdk.context.json to cache AZ lookups (avoid API calls) - ✅ Configure DefaultStackSynthesizer properly - ✅ Use .env files for environment-specific values - ✅ Address CDK warnings during development, not later - ✅ Version control cdk.context.json for team consistency #### 4.3 Resource References & Dependencies (2 commits) **Problem:** Properly referencing resources across stacks. | Commit | Issue | Resolution | |--------|-------|------------| | #24 | Cross-stack references for storage resources | Export/import pattern | | #40 | Secret reference syntax errors | Proper CDK secret constructs | **Resolution Strategy:** - ✅ Use `stack.export()` and `Fn.importValue()` for cross-stack refs - ✅ Pass resources as constructor parameters when possible - ✅ Use proper CDK constructs (Secret.fromSecretArn) instead of raw ARNs - ✅ Define dependencies explicitly with `addDependency()` - ✅ Document cross-stack dependencies in README --- ## 5. Docker/Container Pitfalls (15 commits) ### Key Learnings - Build performance matters significantly for iteration speed - Multi-stage builds reduce image size and build time - User permissions are tricky with shared volumes - Base image selection impacts complexity ### Issues Encountered #### 5.1 Build Performance (3 commits) **Problem:** Docker builds were extremely slow (4+ minutes). | Commit | Issue | Resolution | |--------|-------|------------| | #8 | Got docker building faster with UV | 4 min → 10 sec improvement | | #8 | Implemented multi-stage builds | Separate builder and runtime stages | | #39 | Simplified to leverage base image UV | Use apache/airflow's built-in tools | **Resolution Strategy:** - ✅ Use UV package manager for 25-50x faster pip installs - ✅ Multi-stage builds: builder stage + lean runtime stage - ✅ Copy pre-built virtual environment from builder - ✅ Leverage capabilities of base image (apache/airflow has UV) - ✅ Cache layers effectively by ordering Dockerfile commands **Impact:** Build time: 4 minutes → 10 seconds (24x improvement) #### 5.2 Image Complexity (4 commits) **Problem:** Over-complicated Docker image with unnecessary scripts. | Commit | Issue | Resolution | |--------|-------|------------| | #38 | Massive cleanup: removed 1057 lines | Removed test/debug scripts | | #39 | Removed airflow.cfg file | Use environment variables only | | #39 | Simplified Dockerfile by 16 lines | Leveraged base image features | | #6 | Temporarily removed Docker files | Clean slate approach | **Resolution Strategy:** - ✅ Start with minimal Dockerfile extending apache/airflow base - ✅ Avoid custom entrypoint scripts unless absolutely necessary - ✅ Use environment variables instead of config files - ✅ Remove debugging/testing scripts from production image - ✅ Keep Dockerfile under 50 lines total #### 5.3 Local Development (3 commits) **Problem:** Need to test Docker images locally before ECS. | Commit | Issue | Resolution | |--------|-------|------------| | #9 | Created Docker Compose for local testing | Full local environment | | #10 | Got local dev working | Validated configuration locally | | #26 | Used local deployment to fix ECS issues | Local testing found issues | **Resolution Strategy:** - ✅ Create docker-compose.yml matching ECS configuration - ✅ Test all configuration locally before deploying to ECS - ✅ Use same environment variables in both environments - ✅ Validate database connectivity, secret injection, volumes locally - ✅ Significantly reduces cloud debugging time and cost #### 5.4 Base Image & Dependencies (3 commits) **Problem:** Managing Python dependencies and base image. | Commit | Issue | Resolution | |--------|-------|------------| | #7 | Added build\_and\_[push.sh](http://push.sh) for ECR | Automated image publishing | | #8 | Used UV for dependencies | Faster installs | | #15 | 'aws' CLI not found in image | Base image didn't include AWS CLI | **Resolution Strategy:** - ✅ Use official apache/airflow:3.1.0 as base image - ✅ Only install additional dependencies if truly needed - ✅ Use UV or pip with --no-cache-dir to minimize layer size - ✅ Create automated build/push scripts for CI/CD - ✅ Tag images with git commit SHA for traceability #### 5.5 Sidecar Containers (2 commits) **Problem:** Running git-sync as sidecar with proper image. | Commit | Issue | Resolution | |--------|-------|------------| | #20 | Added git-sync image to ECR | Mirrored [k8s.gcr.io/git-sync](http://k8s.gcr.io/git-sync) | | #50 | Created GitLab CI for git-sync mirroring | Automated mirror updates | **Resolution Strategy:** - ✅ Mirror external images (k8s git-sync) to ECR for reliability - ✅ Use sidecar pattern for orthogonal concerns (git sync, monitoring) - ✅ Configure shared volumes between main and sidecar containers - ✅ Automate image mirroring with CI/CD pipelines - ✅ Document sidecar image sources and update procedures --- ## 6. Infrastructure as Code Best Practices Learned ### From This Journey 1. **Iterative Development is Key** - 54 commits show the value of small, incremental changes - Each commit isolated a specific problem - Easier to rollback and debug 2. **Local Testing Saves Time & Money** - Docker Compose for local validation (commits #9, #10, #26) - Reduced ECS debugging iterations - Faster feedback loop 3. **Separate Stateful from Stateless** - Storage stack (VPC, RDS, Cache) - commit #24 - Service stack (ECS, ALB) can be destroyed/recreated quickly - Independent lifecycle management 4. **Secrets Management** - Never hardcode secrets - Use Secrets Manager for all sensitive data - Proper URL encoding for special characters 5. **Start Simple, Add Complexity** - Initial "AI slop" was too complex - Simplified over time (commits #38, #39) - Remove before adding 6. **Documentation as You Go** - Git commit messages tell the story - Document pitfalls immediately - Future you will thank present you --- ## 7. Recommended Development Workflow Based on lessons learned from 54 commits: ### Phase 1: Foundation (Do This Right First) 1. ✅ Set up CDK project with proper structure 2\. ✅ Create storage stack (VPC, RDS, Cache) 3\. ✅ Configure secrets in Secrets Manager 4\. ✅ Set up cdk.context.json with AZ mappings 5\. ✅ Create .env for environment config ### Phase 2: Docker Development 1. ✅ Create minimal Dockerfile extending apache/airflow 2\. ✅ Set up docker-compose.yml for local testing 3\. ✅ Test all Airflow services locally 4\. ✅ Validate secrets, DB connection, volumes 5\. ✅ Optimize build with UV/multi-stage ### Phase 3: AWS Deployment (Incremental) 1. ✅ Deploy storage stack first 2\. ✅ Push Docker image to ECR 3\. ✅ Deploy webserver only (with ALB) 4\. ✅ Deploy scheduler 5\. ✅ Deploy triggerer 6\. ✅ Deploy worker last (with auto-scaling) ### Phase 4: Optimization 1. ✅ Enable health checks after services stable 2\. ✅ Add circuit breaker 3\. ✅ Configure auto-scaling 4\. ✅ Add Spot instances for workers 5\. ✅ Set up monitoring and alarms ### Phase 5: Production Hardening 1. ✅ Enable secret rotation 2\. ✅ Configure backup policies 3\. ✅ Set up CI/CD pipelines 4\. ✅ Document runbooks 5\. ✅ Load testing and tuning --- ## 8. Cost Optimization Insights ### What Worked - **Spot Instances for Workers** (commit #45): 50-70% cost savings - **ARM64 Architecture** (commits #13, #14): Better price/performance - **Separate Stacks** (commit #24): Can destroy/recreate services without affecting storage - **Auto-scaling** (commit #51): Only pay for capacity you use ### What Didn't Work - **EFS** (commits #15-18): More complex and costly than git-sync - **Always-on development**: Could use dev/prod environment separation --- ## 9. Key Metrics ### Development Journey - **Total Commits:** 54 - **Major Refactors:** 3 (commits #4, #24, #38) - **Pitfall Fixes:** 47 - **Days of Development:** ~10 days (Oct 28 - Nov 6, 2025) ### Technical Debt Resolved - **Deleted Lines:** ~6,000+ (cleanup commits #38, #39, #52) - **Documentation Added:** 2,500+ lines - **Build Time Improvement:** 24x faster (4 min → 10 sec) ### Final Architecture - **Stacks:** 2 (storage, service) - **ECS Services:** 4 (webserver, scheduler, worker, triggerer) - **Supporting Services:** 3 (RDS, Valkey, git-sync) - **Auto-scaling:** 1-10 workers based on load - **Spot Instances:** Enabled for workers --- ## 10. Top 10 Lessons Learned 1. **Test Locally First** - Docker Compose saved countless hours debugging in AWS 2\. **URL-Encode Passwords** - Special characters in passwords will break everything 3\. **Use Git-Sync, Not EFS** - Simpler, cheaper, more reliable for DAG deployment 4\. **Separate Stateful/Stateless** - Independent lifecycle = faster iteration 5\. **UV Package Manager** - 24x faster Docker builds 6\. **Start Simple** - Remove complexity before adding features 7\. **Security Groups Matter** - Document all connectivity requirements 8\. **Health Check Grace Periods** - DB-dependent services need 300-600 seconds 9\. **Spot Instances for Workers** - 50-70% cost savings, minimal impact 10\. **Commit Often** - Small commits make debugging and rollback easier --- ## 11. If Starting Over, Do This ### Skip These Mistakes - ❌ Don't use static airflow.cfg files - ❌ Don't start with all services at once - ❌ Don't use EFS for DAGs (use git-sync) - ❌ Don't ignore CDK warnings - ❌ Don't skip local testing - ❌ Don't generate passwords with special characters (or URL-encode them) - ❌ Don't use complex entrypoint scripts ### Do These Things - ✅ Start with storage stack + webserver only - ✅ Use UV for fast Docker builds from day 1 - ✅ Set up docker-compose.yml immediately - ✅ Use environment variables exclusively - ✅ Configure Spot instances from the start - ✅ Document as you go - ✅ Use git-sync for DAG deployment ### Time Saved Following this guide could reduce development time from **~80 hours to ~20 hours**, avoiding the 47 pitfall commits. --- ## Conclusion This journey through 54 commits demonstrates the iterative nature of cloud infrastructure development. The key insight is that **failure is part of the learning process**, and each commit represents a lesson learned. The most valuable outcome isn't just a working Airflow deployment, but the understanding of: - How ECS services communicate - Why certain patterns work (git-sync) and others don't (EFS) - The importance of local testing - How to structure IaC for maintainability These lessons are transferable to any ECS/Fargate deployment, not just Airflow. **Final Architecture Success Rate:** 100% (after 54 commits and ~47 fixes) **Would Do It Again:** Yes, with this document as a guide --- ## Learning about Rust Benchmarking with Sudoku from 5 minutes to 17 seconds **URL**: https://smiling.dev/blog/rust-benchmarking-sudoku-5min-to-17sec **Author**: Bryson **Published**: 10/1/2025 **Updated**: 10/1/2025 **Tags**: Rust, Software Engineering, Criterion, Benchmarking **Description**: I'll take you through the process of optimizing a Sudoku solver written in Rust. We'll start with a simple, unoptimized version and apply a series of optimizations that will take the time to solve 100,000 puzzles from over 5 minutes down to just 33 seconds, and 20,000 of the hardest puzzles from over 2 minutes down to just 17 seconds. I’ll take you through the process of optimizing a Sudoku solver written in Rust. We’ll start with a simple, unoptimized version and apply a series of optimizations that will take the time to solve 100,000 puzzles from over 5 minutes down to just 33 seconds, and 20,000 of the hardest puzzles from over 2 minutes down to just 17 seconds. ### The Setup The project is a command-line Sudoku solver written in Rust. The puzzles are read from text files in the `src/puzzle_banks` directory, thanks to the github project [Sudoku Exchange Puzzle Bank](https://github.com/grantm/sudoku-exchange-puzzle-bank). Each line in these files represents a single puzzle in the following format: ``` 0000847b216e 020900000048000031000063020009407003003080200400105600030570000250000180000006050 2.3 ``` The second part of the line is the puzzle itself, where `0` represents an empty cell. The basic flow of the program is: 1. Read the puzzle file. 2. For each line, parse the puzzle string into a `Board` struct. 3. Call the `solve` function on the board. 4. Print the statistics. ### The Benchmarks Here is a summary of the benchmarks for both the “Easy” and “Diabolical” puzzle sets: ![](https://cdn-images-1.medium.com/max/800/1*TUjpjLPm1uBDM_XdCdtfow.png) ### The Initial State: A Simple Backtracking Solver Our starting point was a straightforward backtracking solver. The solver would recursively try to place valid numbers in empty cells. If a path led to a dead end, it would backtrack and try a different number. While this approach worked, it was slow. The initial benchmark for solving 100,000 easy puzzles was **5 minutes and 26 seconds**, and 20,000 of the hardest puzzles took **2 minutes and 18 seconds**. The main culprits for this slow performance were: * **Excessive Cloning**: The board state was cloned for every single step of the recursion. * **Frequent Allocations**: Functions that fetched rows, columns, and sections of the board were returning new vectors (`Vec`s), leading to frequent memory allocations. Here is what the initial `solve` function looked like: ``` pub fn solve(mut board: Board) -> Result { // Find the first empty cell for row in 0..board.height() { for col in 0..board.width() { if board.get_cell(row, col).is_some() { continue; } // Get valid options for this cell let options = board.get_options(row, col); if let Some(option_values) = options { // Try each valid option for val in option_values { // Place the value board.set_cell(row, col, Some(val)); // Recursively try to solve the rest match solve(board.clone()) { // The expensive clone! Ok(solved) => return Ok(solved), // Found solution! Err(_) => { // This path didn't work, backtrack board.set_cell(row, col, None); // Continue to next option } } } // No valid options worked, backtrack return Err("No valid solution from this state".to_string()); } else { // No options available for this empty cell - dead end return Err("No options available".to_string()); } } } // If we get here, all cells are filled - puzzle is solved! Ok(board) } ``` ### First Optimization: Reducing Allocations and Clones The first step was to reduce the memory overhead. We did this by: 1. **Removing** `board.clone()`: Instead of cloning the board for each recursive call, we modified the `solve` function to use a mutable reference (`&mut Board`). This single change had a huge impact on performance. **Before:** ``` match solve(board.clone()) { ... } ``` **After:** ``` if solve(board).is_ok() { ... } ``` 1. **Using Iterators**: The `get_row`, `get_col`, and `get_section` functions were refactored to return iterators instead of new vectors. This avoided unnecessary memory allocations. 2. **Before:** ``` pub fn get_row(&self, row: usize) -> Vec> { ... self.cells[start..end].to_vec() } ``` 1. **After:** ``` pub fn get_row(&self, row: usize) -> &[Option] { ... &self.cells[start..end] } ``` 1. **Optimizing Option Lookups**: The `get_options` function, which finds valid numbers for a cell, was optimized to use a boolean array for faster lookups, instead of searching through vectors. 2. **Before:** ``` let options: Vec = (1..=self.max_value.get()) .filter_map(|u| NonZeroU8::new(u)) .filter(|x| { !row_values.contains(x) && !col_values.contains(x) && !section_values.contains(x) }) .collect(); ``` 1. **After:** ``` let mut used = vec![false; self.max_value.get() as usize + 1]; // ... populate used array ... let options: Vec = (1..=self.max_value.get()) .filter_map(|u| { if used[u as usize] { None } else { NonZeroU8::new(u) } }) .collect(); ``` These changes resulted in a **21% speedup** on the easy puzzles and a **55% speedup** on the diabolical puzzles. This shows that reducing memory allocations is even more important for harder problems, likely because the recursion depth is much greater, and the overhead of cloning larger and more complex board states adds up. **Criterion** I really wanted to use Criterion for my first time to better get a view into how each puzzle is being solved. Here is a set up of criterion running a single puzzle 5000 times with its code. Notice that each cycle is around 1.25 ms. ``` use criterion::{black_box, criterion_group, criterion_main, Criterion}; use sudoku_solver::{Board, solve}; ``` ``` fn criterion_benchmark(c: &mut Criterion) { let puzzle = "050703060007000800000816000000030000005000100730040086906000204840572093000409000"; c.bench_function("solve easy", |b| { b.iter(|| { let mut board = Board::from_puzzle_bank_line(&format!("id {} 1.2", puzzle)).unwrap(); solve(black_box(&mut board)) }) }); } ``` ``` criterion_group!(benches, criterion_benchmark); criterion_main!(benches); ``` ![](https://cdn-images-1.medium.com/max/800/1*PWmmiB_iCno4g6mPK6KAdQ.png) Criterion Benchmarks for the first ### Parallelism with Rayon The next step was to introduce parallelism using the `rayon` crate. ### Attempt 1: Parallelizing the Solver Our first attempt was to parallelize the solver itself. The idea was to explore different solution paths in parallel. However, this resulted in a **9% slowdown** on the easy puzzles and a staggering **98% slowdown** on the diabolical puzzles. The overhead of managing parallel tasks and the re-introduction of some cloning outweighed the benefits, especially for the harder puzzles which have a deeper recursion space and require more coordination between threads. ``` pub fn solve(board: &mut Board) -> Result<(), String> { // ... let solution = options .into_par_iter() // Parallel iteration .find_map_any(|val| { let mut new_board = board.clone(); // Cloning is back! new_board.set_cell(row, col, Some(val)); if solve_sequential(&mut new_board).is_ok() { Some(new_board) } else { None } }); // ... } ``` And criterion shows the same regression. I think the skew and larger variance in the benchmark is the most noticable that we are hitting non-deterministic data sharing between the threads. **Turns out that parallel code can easily make your code slower if you’re not thinking.** ![](https://cdn-images-1.medium.com/max/800/1*GCQNMGkKHAJS4G7PcR1Itw.png) ### Attempt 2: Parallelizing the Puzzle Set The second, more successful approach was to solve multiple puzzles in parallel. We used `rayon` to distribute the puzzles from the input file across multiple threads. Each thread would then solve one puzzle sequentially. ``` use rayon::prelude::*; fn main() { // ... let results: Vec> = contents .par_lines() // Parallel iteration over lines .filter(|line| !line.trim().is_empty()) .map(|line| { let mut board = Board::from_puzzle_bank_line(line)?; solve(&mut board) }) .collect(); // ... } ``` This “embarrassingly parallel” approach was very effective, resulting in a **56% speedup** on the easy puzzles and a **52% speedup** on the diabolical puzzles. The benchmark iteration is looking much better. Something to note is that the time to solve the puzzle has not changed much since the first solution, but we are using parallelism of the cpu cores better. ![](https://cdn-images-1.medium.com/max/800/1*62j3I8Z1kDMh55ddugYNuA.png) ### The Big Breakthrough: Constraint Propagation The most significant performance gain came from improving the solver’s logic. Instead of immediately resorting to backtracking, we first added a logical deduction step. We implemented an `eliminate` function that repeatedly scans the board for "naked singles" – empty cells that have only one possible valid number. When it finds one, it fills it in. It keeps scanning and filling until it can't find any more naked singles. ``` fn eliminate(board: &mut Board) -> bool { let mut made_change = true; while made_change { made_change = false; for row in 0..board.height() { for col in 0..board.width() { if board.get_cell(row, col).is_some() { continue; } if let Some(options) = board.get_options(row, col) { if options.len() == 1 { board.set_cell(row, col, Some(options[0])); made_change = true; } } } } } // ... check if solved ... } ``` The main `solve` function now first calls `eliminate`. Only if the puzzle is not solved by logic alone does it proceed to the backtracking algorithm. ``` pub fn solve(board: &mut Board) -> Result<(), String> { if eliminate(board) { return Ok(()); } solve_backtracking(board) } ``` This is a form of **constraint propagation**, and it dramatically prunes the search space for the backtracking solver. This change brought the time down to an incredible **33 seconds** for the easy puzzles (a **74% improvement**) and **17.8 seconds** for the diabolical puzzles (a **69% improvement**). The benchmark now shows that the new improvements to the algorithm is significantly improving the time to solve a single puzzle, changing from 1.2 ms to 46 µs.  ![](https://cdn-images-1.medium.com/max/800/1*TBi8QWDyht0J3q9f8h8RRw.png) It’s important to clarify a common point of confusion: this optimization was not about “backtracking and not recursion.” Recursion is a programming technique (a function calling itself), while backtracking is an algorithm that uses trial and error. Our solver still uses both, but the addition of the `eliminate` function means it has to do much less backtracking. ### Final Touches: Protecting the Puzzle As a final improvement, we added a safeguard to prevent the original puzzle’s numbers from being accidentally modified. This makes the solver more robust and protects the integrity of the puzzle. ``` // In the Board struct pub struct Board { cells: Vec>, initial_cells: Vec, // New field // ... } // In the set_cell function pub fn set_cell(&mut self, row: usize, col: usize, val: Option) { let idx = self.width * row + col; if self.initial_cells[idx] { panic!("Cannot change initial cell"); } // ... } ``` ### Conclusion This journey of optimization highlights several key principles of writing high-performance code: * **Choose the right data structures and algorithms**: The move from `Vec`s to iterators and the introduction of the `eliminate` function had the 2nd biggest impact. * **Profile your code**: Understanding where the bottlenecks are is crucial for effective optimization. * **Parallelism is not a silver bullet**: The overhead of parallelism can sometimes make your code slower. It’s important to choose the right parallelization strategy for your problem. * **A little logic can save a lot of time**: The simple `eliminate` function provided a massive performance boost by reducing the search space for the more expensive backtracking algorithm. Small Cloning made a small difference, but improving the number of times to scan the puzzle was more important. * **In further optimizations**, I’d want to eliminate much of the iteration over the table, calling `get_options()` for the entire board first, then using those for a speedy test and insert. By applying these principles, we were able to take our Sudoku solver from a slow, 5-minute process to a highly optimized solver that can crack 100,000 easy puzzles in just 33 seconds, and 20,000 of the most diabolical puzzles in under 18 seconds.  [Github Repo](https://github.com/Bryson14/RustLangFun/tree/main/sudoku-solver) --- ## Choosing Intentional Media Over Doom Scrolling **URL**: https://smiling.dev/blog/choosing-intentional-media-over-doom-scrolling **Author**: Bryson **Published**: 9/23/2025 **Updated**: 9/23/2025 **Tags**: Social Media, Information, AI **Description**: Being intentional about the sources of media, articles, entertainment with my time has been helping me reflect on the current state of technology and social media apps. I'm wary of the amount of time I use it and the image that I present to kids. Its a weird world we live in. I can know about everything that is happening in the world and yet I feel like I trust so little. I'm sure it is happening to everyone. I see podcasts, video shorts, long form videos, coworker conversations all pointing to that we are getting so much information but we hold on to so little. I get this scenario a lot. I get a text message from a family member asking if I have seen the Instagram video they sent me. I haven't. I'm intentionally kid of avoiding it. I then see a notification pop up that another family member has sent me a Facebook video over Messager. I get home from work, and I now have 20+ notifications of Instagram videos, reddit threads, x word-fights, and Facebook garbage on my phone. I'm tired just thinking of it. So what should I do? I've considered for a few minutes getting a flip phone, but i need email and most regular apps for banking. --- ## Postgres and Supabase beat AWS Amplify 9 times out of 10 **URL**: https://smiling.dev/blog/postgres-and-supabase-beat-aws-amplify-9-times-out-of-10 **Author**: Bryson **Published**: 8/18/2025 **Updated**: 8/18/2025 **Tags**: Postgres, Supabase, AI Agents **Description**: Don’t try to bring disjointed technologies and a half made developer experience into your next business. Use reliable tools and well designed products # Solid Technology Makes for Good Choices A few months ago I wrote an article called why AWS amplify might be the best backend framework. I wrote it partly as a way to test my own knowledge of what I’ve done for the last 2 1/2 years. Also, because there was a little bit of truth in the goodness of that framework. However, I am going back on my thoughts, and I am now walking away from AWS amplify for Supabase. The TL;DR is that using a reliable and flexible, database (Postgres) has dramatically improved the performance of the website, the developer experience and maintenance. Consolidating all back and functions into a single web server (Hono) using typescript has sped up development and improved errors. so I guess I’m going to be addressing two different audiences here. The first audience is those that are what reading this and genuinely trying to understand what they should build their next app with. The second audience is those that are gonna say, well, of course you should use a sequel database. Why did you start with dynamite DB in the first place? The second audience is partly me now after going through this. ## Project Context For the last 2 1/2 years, I’ve been making and maintaining a tutoring website. If you wanna give it a check, go to tutorthefuture.com. It started out as massapequatutor.com, and then went to a rebrand, and now it’s even going through another rebrand. The original code that I inherited had a particular data model and was using dynamo DB. So over 2 1/2 years ago, I saw a W simplify as another system using dynamite DB, on AWS, and I thought to myself wow this is great. This will help me with all of my data and API needs. The promise of AWS simplify is neat, it tries to be a fire base competitor. But ultimately, I think it’s use of Graf QL and Dynamo DB ends being the foot gun. I think they realize this and they have even started going to a AWS amplify Jen two, which is really leaning into all type scripts, and more quick development times. But I still think underneath their storage layer just causes too many issues. ## Data and storage After going through this, I’ve realized that the heart of most applications is the data. I think someone has said multiple times that most applications at their heart are just a database on under a pretty UI. And in this case, it was very true. The amount of times I found myself fighting with the data set up and dynamite. DB was terrible. I tried both leaning into the data duplication and just using no sequel. I tried making the database feel more like SQL, and then doing my own application layer joining over data. But in both cases, it was just terrible. The application code couldn’t guarantee what the data was, also it ended up being a bunch of waterfall network request to get the data that I needed. they use Graf QL an app sync to try and fix that so you can pull multiple pieces of data together, but it only works really for one to one relationships. And even then the performance was just bad. Additionally, dynamite DB only has a one megabyte page limit. So then I was always overthinking how to deal with Paige nation. And some places like admin dashboard where you needed to do different queries and aggregations, I just bit the bullet and I would download all the database from the system, which fortunately only ever got to around 5000 records for the biggest table. And then I would run the aggregations locally on the browser. And I knew it was bad, and maintaining it was bad, and every time the client asked for a different aggregation, or view, or when it update the data. I found myself always fighting The database. I found myself fighting how I was saving data, and making things faster, and also just really making things easier for me to logically understand. ## PostgreSQL so let’s jump forward to Postgres. I really haven’t worked much with it in my career, so it was a little bit of a learning curve. But I’m very happy that I’ve taken the time to understand it and to get more familiar with how SQL works. It took me probably two weeks to ramp up and learn about the basics of SQL, super bases, and more particularly all the migrations that you can do with SQL. But now it feels so much more manageable. Also the PostgREST API layer that super base uses is just beautiful. It’s so awesome that I can do complex queries just for my client and I’m getting exactly the day that I want. And then if I really do need more complex functions or permission, scoped data, I can drop-down into RLS or database functions that have their own properties and permissions. It’s like someone has put 40 years of thought into these database designs. ## AWS Bubble I was so dumb in hindsight is that I was trying to do joints in dynamo GB, but it ended up being this waterfall disaster of network request. And so in some of the tables, I tried to get around this by just adding everything into the table, similar to how you would de normalize everything in mango DB. But that just ended up being terrible as well. So it is so great now that I can do native joints and everything just magically pulls together. Perhaps this is because many woworkers and colleages around me are only familiar with AWS. They have only worked with AWS or even have worked at AWS. They are very heavy on the web-scale No-SQL must-use-dynamodb-db-or-else retoric. Clearly AWS has something good going for them, but they also have a much different set of challenges and resources compared to a single person consultancy. --- ## Jump on the UV Python Bandwagon. You Won’t Be Disappointed. **URL**: https://smiling.dev/blog/jump-on-the-uv-python-bandwagon-you-wont-be-disappointed **Author**: Bryson **Published**: 7/1/2025 **Updated**: 7/1/2025 **Tags**: UV, Python, Software Development, Modern Tooling **Description**: UV fixes a lot of common headaches for Python developers. It helps you ditch old habits like requirements.txt (which was never really meant to be a standard) and the hassle of activating virtual environments. With UV, your projects become cleaner, more reliable, and a lot more fun to work with. Tired of slow `pip` installs, confusing virtual environments, and messy `requirements.txt` files? Meet **UV**, the new Python package installer and manager that's changing the game. UV is super fast, easy to use, and much better than older tools like `pip`, `virtualenv`, and even `poetry`. I’ve started to use it and forget about all the headaches and duplicate python installs I’ve had. Fancy new [MCP projects](https://glama.ai/mcp/servers/@ReAPI-com/mcp-openapi#:~:text=of%20API%20development.-,Cursor%20Configuration,-To%20integrate%20the) and [common data validation libraries](https://github.com/pydantic/pydantic) are using it because it just makes sense. Just look for a uv.lock file in your favorite python repo. UV fixes a lot of common headaches for Python developers. It helps you ditch old habits like `requirements.txt` (which was never really meant to be a standard) and the hassle of activating virtual environments. With UV, your projects become cleaner, more reliable, and a lot more fun to work with. This article will show you the best parts of UV and why you should start using it today. ![Happy Programmer with Python again](../images/jump-on-uv-bandwagon-python-hugger.png) ## 1. Get Started Fast: How to Install UV Installing UV is quick and simple. You can usually get it running with [just one command](https://github.com/astral-sh/uv?tab=readme-ov-file#installation): Once it’s installed, you’ll immediately notice how much faster UV is compared to what you’re used to. Also note that the preferred installation is a standalone binary. This helps it to be outside of the traditional python path issues where py shims, pypath, and other alias paths have made python versions and binaries a tougher-than-necessary installation. ## 2. Cleaner Scripts: No More `requirements.txt` for One-Offs, Hello PEP 723 and `pyproject.toml` For years, Python developers have relied on `requirements.txt` to list their project's dependencies. The dirty little secret I found out a few months ago is `requirements.txt` was never meant to be a standard. It was just an implementation detail created by the `pip` team that accidentally became the norm. This often led to messy dependency management, especially for larger applications. ### The Modern Python Way: `pyproject.toml` (PEP 621, since 2020) For full-fledged applications, multi-file projects, and libraries, the official and recommended way to declare your project’s dependencies is using `pyproject.toml`, specifically following **PEP 621**. This standard has been around since 2020 and provides a clear, structured, and central place for all your project's metadata, including its dependencies. UV fully supports `pyproject.toml` for managing your application's dependencies. It even has development dependancies all set. I worked for a company that has multiple files and a paragraph how to use them in README. Getting `requirements.txt` , `requirements-dev.txt` , and `requirements-async.txt` if a symptom of a system that needs to evolve. ### Example `pyproject.toml` for an application: ```toml # pyproject.toml [project] name = "my-awesome-app" version = "0.1.0" description = "A multi-file Python application" dependencies = [ "requests>=2.30", "fastapi~=0.100", "uvicorn[standard]", ] ``` UV will read this file to understand and install your application’s dependencies, making `requirements.txt` obsolete for these kinds of projects. ### The UV Way for Single-File Scripts: PEP 723 While `pyproject.toml` is excellent for applications, what about those quick, self-contained Python scripts you write for automation, data processing, or little utilities? This is where **PEP 723** comes in, and UV embraces it perfectly. I’ve used it for lots of data processing, database migrations, and more. **The Problem with Old Ways for Scripts.** Sharing a single Python script often meant you also had to provide a `requirements.txt` file and explain how to set up a virtual environment and install dependencies. It was extra work for something that should be simple. **The UV Solution with PEP 723.** UV, working with PEP 723, lets you embed your script's dependencies directly within the script itself using a special comment block. This makes your single-file scripts truly self-contained and super easy to share. This is ideal for "one-off" scripts, helper utilities, or anything that doesn't need a full `pyproject.toml` setup. **Here’s how it looks for a simple script:** ```python # /// script # dependencies = ["requests", "beautifulsoup4"] # /// import requests from bs4 import BeautifulSoup def get_page_title(url): response = requests.get(url) soup = BeautifulSoup(response.text, 'html.parser') return soup.title.string if soup.title else "No title found" if __name__ == "__main__": url = "https://www.example.com" print(f"Title of {url}: {get_page_title(url)}") ``` **How it works:** When you run this script with `uv` (like `uv run your_script.py`), UV automatically finds the `dependencies` block. It then quickly sets up a temporary environment, installs the listed packages, and runs your script. No more `venv` commands or `pip install` steps for simple scripts! This clean separation means you use the right tool for the job: `pyproject.toml` for applications and PEP 723 for your handy one-off scripts. I’ve even seen someone put shebang lines at the top, making this file executable from a bash settings. [Article Link](https://github.com/astral-sh/uv?tab=readme-ov-file#installation). This now makes it a possible replacement for small scripts written in go or rust because they were simple and flexible executable files. ## 3. Supercharge Your CI/CD: Fast and Cached Builds with GitLab ![Fast and Slow](../images/jump-on-uv-bandwagon-fast-slow.webp) [Source](https://media.licdn.com/dms/image/v2/C5612AQGtYBNvl_yiYQ/article-cover_image-shrink_600_2000/article-cover_image-shrink_600_2000/0/1651676443940?e=2147483647&v=beta&t=FhAKMIVHW_PAMInZDoDJ3R7TMB_xMKYi279M94rLDNI) UV truly shines in automated testing and deployment setups like GitLab CI/CD. Its speed and smart caching can drastically cut down your build times. UV is designed to be fast since it uses a single-threaded [tokio runtime](https://docs.rs/tokio/latest/tokio/). The uv pip install command is optimized for speed and works great with caching. You don’t even need to make a virtual environment yourself; UV handles it. Here’s an updated GitLab CI/CD example using the latest UV image and caching tips: [Source Docs](https://docs.astral.sh/uv/guides/integration/gitlab/) ```yaml # .gitlab-ci.yml (UV way) variables: UV_VERSION: "0.5" # Make sure this matches the latest UV version PYTHON_VERSION: "3.12" BASE_LAYER: bookworm-slim UV_LINK_MODE: copy # Needed for GitLab CI's build directory setup UV_CACHE_DIR: ".uv-cache" # Where UV stores its cache stages: - build - test uv-install: stage: build image: ghcr.io/astral-sh/uv:$UV_VERSION-python$PYTHON_VERSION-$BASE_LAYER cache: # Use uv.lock to make sure the cache changes when your dependencies change key: files: - uv.lock paths: - $UV_CACHE_DIR # Cache UV's internal package cache - .venv/ # Cache the virtual environment created by UV policy: pull-push script: # This command creates or updates the virtual environment # and installs dependencies from uv.lock (or pyproject.toml if no lock) - uv sync # Clean up the cache to save space - uv cache prune --ci run_tests: stage: test image: ghcr.io/astral-sh/uv:$UV_VERSION-python$PYTHON_VERSION-$BASE_LAYER # Make sure this job can use the cache from the 'uv-install' job needs: - uv-install script: # Run your tests using uv run, which uses the managed environment - uv run pytest ``` ### What this means for your CI/CD: - **Faster Builds:** UV's installer is super quick, making your CI/CD jobs run much faster. - **Smart Caching:** By caching `$UV_CACHE_DIR` and `.venv/` based on `uv.lock`, your pipeline runs will be lightning fast after the first one, especially for small code changes. - **Simpler Setup:** Your CI/CD script is cleaner because UV takes care of virtual environment management behind the scenes. - **Reliable Results:** UV's lock files (we'll talk about these next) ensure your CI/CD environment always uses the exact same package versions you tested locally. ## 4. Secure Private Repositories with `.netrc` Dealing with private Python packages can be annoying, often requiring you to put sensitive details directly in your code or in environment variables. UV has a better way: it works smoothly with `.netrc` files. **The Problem:** Hardcoding usernames and passwords or using temporary environment variables for private package servers. **The UV Solution:** UV uses your `~/.netrc` file. Just add an entry for your private package server, and UV will automatically use those login details when it installs packages from there. ### Example `~/.netrc` entry: ```text machine acme.gitlab.com login oauth2 password thekrusty-krab-pizza-is-the-pizza-for-u-&-me ``` **How it works:** When UV needs a package from `your-private-repo.com` (as listed in your `pyproject.toml` or `uv.lock`), it will look up the login details in your `.netrc` file. This keeps your login info secure and out of your project files. ## 5. Helpful Tools: `uvx` for Ruff, Ty, and More! UV isn’t just about managing packages; it also has `uvx` to make your daily coding easier. `uvx` lets you run tools like linters and type checkers within a UV-managed environment without installing them globally. ### `uvx ruff` for Quick Code Checks and Fixing Ruff is known for being an incredibly fast tool for checking and formatting Python code. `uvx ruff` makes it even better by running ruff with the exact versions needed for your project, without messing up your global Python setup. ```bash uvx ruff . # Check your whole project for style issues uvx ruff --fix . # Check and automatically fix style issues ``` This is super helpful for making sure everyone on your team follows the same code style. ### Type Checking with `uvx ty` or `uvx mypy` While ruff handles code style, `uvx typ` (or `uvx mypy` for MyPy) helps you catch potential errors before you run your code. This leads to stronger, easier-to-maintain code. Overall, `typ` has helped a bit, but it's far from being as comprehensive as MyPy currently. ```bash uvx ty check your_module.py # Run type checks on a specific file uvx ty check . # Run type checks on your whole project ``` The great thing about `uvx` is that you don't need to manually `pip install ruff` or `mypy` in every virtual environment. `uvx` takes care of setting up these tools on the fly, making them instantly ready for your project. ## 6. Solid Projects with Lock Files UV strongly supports **lock files**. These files (often named `uv.lock`) pinpoint the exact version of _every single package_ your project needs, including those that your direct dependencies rely on. ### Why Lock Files are a Big Deal: - **Reliable Projects:** Everyone working on the project, and your CI/CD systems, will use the exact same package versions. This stops the common "it works on my machine!" problem. - **Stable Updates:** Your project won't break if a hidden dependency suddenly updates. UV makes creating and using lock files simple: This way, your `pyproject.toml` lists the general packages you want, and `uv.lock` records the _exact_ versions that actually get installed. ## 7. Migrating Scripts: Boto3 and Async Boto3 with UV UV is perfect for managing dependencies for migration scripts or other helper tools that interact with cloud services. Let’s say you need `boto3` (for AWS services) and `aioboto3` (for async AWS calls) for a database migration script. Here’s how you’d set up a script that uses them with UV ```python # db_migration_script.py # /// script # dependencies = ["boto3", "aioboto3"] # /// import asyncio import boto3 import aioboto3 async def perform_async_db_backup(bucket_name, object_key, data): async with aioboto3.client("s3") as s3_client: await s3_client.put_object(Bucket=bucket_name, Key=object_key, Body=data) print("Got it coach") def perform_sync_db_cleanup(table_name): dynamodb = boto3.resource('dynamodb', region_name='us-east-1') # Adjust region as needed table = dynamodb.Table(table_name) print("Aye aye captain") async def main(): await perform_async_db_backup("your-backup-bucket", "db_dump_20250701.json", b"some json data") perform_sync_db_cleanup("your-database-table") if __name__ == "__main__": asyncio.run(main()) ``` **How it helps:** * **Self-contained Migrations:** The script itself tells UV what it needs (`boto3`, `aioboto3`). When you run `uv run db_migration_script.py`, UV makes sure these are available. * **Easy Sharing:** Just share the script! No separate `requirements.txt` for this specific helper. * **Consistent Environments:** Whether you run it locally or in your CI/CD, UV creates the correct environment, preventing issues from missing packages. This is super important for critical tasks like database migrations. ## 8. No More Virtual Environment Mess One of the quieter but powerful benefits of UV is how it handles virtual environments. It avoids the confusing “virtual shim/alias hell” that some other tools create. UV manages environments cleanly, making them feel like a natural part of your project, not a separate, tricky layer you constantly have to wrestle with. ![Python Mess](../images/jump-on-uv-bandwagon-python-mess.webp) Source: [https://xkcd.com/1987/](https://xkcd.com/1987/) ## Conclusion: Your Python Workflow, Made Easy UV is more than just a new tool; it’s a huge step forward for Python development. It brings speed, smart caching, modern best practices like PEP 723 and lock files, and handy utilities like `uvx` to the forefront. It tackles many of the annoying problems Python developers have faced for a long time. If you’re still stuck with older ways of managing Python projects, now’s the time to switch. **Jump on the UV Python bandwagon. You won’t be disappointed.** Your development process will be smoother, faster, and much more enjoyable. Comment and clap for the article if you learned something new! Or check out some of my other articles on medium or on https://medium.com/@BryMei --- ## After 3 years of AWS Amplify, is it the Best Backend as a Service? **URL**: https://smiling.dev/blog/after-3-years-of-aws-amplify-is-it-the-best-backend-as-a-service **Author**: Bryson **Published**: 5/12/2025 **Tags**: Backend as a Service, AWS **Description**: As a one-person team managing backend services for a few businesses, finding a reliable and efficient Backend as a Service (BaaS) is crucial. For the past three years, AWS Amplify has been that service for me. ![A sad programmer thinking about the time wasted with python project management and debugging](../images/after-3-years-of-amplify-aws.webp) As a one-person team managing backend services for a few businesses, finding a reliable and efficient Backend as a Service (BaaS) is crucial. For the past three years, AWS Amplify has been that service for me. Given AWS’s reputation for solid infrastructure, my initial assessment remains largely positive: it’s a good platform, albeit with its own specific way of doing things. It has enabled me to deliver features and manage operations effectively, proving to be a powerful tool for a lean team. So, would I use it again? Perhaps, but not without acknowledging some significant points of friction I’ve encountered along the way. ## Infrastructure and the Gen 1 to Gen 2 Transition One of the most significant points of contention has been the slow transition from Amplify Gen 1 to Gen 2. While Gen 2, announced in 2024, promises a more developer-friendly, code-first approach built on AWS CDK templates (a welcome change from the more rigid CloudFormation templates of Gen 1), the lack of a seamless official conversion tool has been a letdown. I’m still reliant on the older tools for my existing projects. Ideally, I’d prefer working with a tool like Terraform, which offers greater flexibility and a clearer infrastructure-as-code approach compared to CloudFormation. I’ve been looking at that CloudFormation console for years, and the nested stacks and rollback features still seem like a mystery to me. While the Gen 2 move towards CDK is a step in the right direction, the delayed and perhaps clunky migration path dampens the excitement. ## Database Challenges: The NoSQL Trap with DynamoDB The database experience with Amplify, primarily leveraging DynamoDB, starts deceptively simply. At first, it’s pretty nice. The Amplify Studio UI allows for easy creation of data models, which initially feels intuitive. However, this interface, and the $index and @connection directives, subtly encourage a relational (SQL) database design mindset. It sounds great, it's a trap. This approach fundamentally clashes with the optimal usage patterns of a NoSQL database like DynamoDB. DynamoDB thrives on access patterns being defined upfront, often requiring denormalization and careful planning of Partition and Sort Keys, along with Global Secondary Indexes (GSIs). Trying to apply a SQL-like schema design to DynamoDB through Amplify’s data modeling tools feels like a trap. You either end up duplicating and nesting data across multiple tables to reduce the need for complex joins (which DynamoDB isn’t built for) or cramming everything into one table and relying on intricate Sort Keys and GSIs for querying. The flexibility of SQL databases like PostgreSQL, allowing for easy pivoting, summing, and aggregation, is something I deeply miss. While Amplify introduced the @searchable directive to push data to an OpenSearch cluster for more flexible searching, this comes at an added cost, of course. My workaround has been using DynamoDB streams and Lambda functions to create aggregated tables for summary data – a pattern that works but is obviously a workaround for the limitations of the database within the Amplify context. Comparing this to the inherent capabilities of a PostgreSQL database in a service like Supabase highlights a significant drawback of the Amplify/DynamoDB pairing for certain use cases. Part of me really wishes I had gone with Supabase because of the PostgreSQL database. ## Cost Considerations ![Cost of the website over over the year](../images/aws-amplify-yearly-costs.png) On the financial front, Amplify has been quite cost-effective for my needs. The cost has been great! Comparing my average monthly bill of around $14 (which includes costs for services like SES for email and SNS for texts) to the $25/month starting tier of Supabase, Amplify certainly seems to win purely on price for my current usage. However, this cost analysis feels a bit petty when considering the development overhead and workarounds necessitated by using DynamoDB instead of a more flexible SQL database like PostgreSQL offered by Supabase. The features and ease of querying a SQL database could potentially justify a higher cost for many projects. ## UI Component Experience: A Disappointing Detour Initially, I drank the Kool-Aid and went all in at first with the AWS Amplify UI components. I won’t do that again. I was drawn by the promise of converting Figma designs and auto-generating forms based on data models. This turned out to cause me so many hours of rewrites. The system felt clunky. ![auto form creation with aws amplify](../images/aws-amplify-form-generations.png) Compared to mature and popular UI libraries like Shadcn, MUI, or Mantine, Amplify UI components fall short. These alternative libraries offer greater flexibility, better performance, and a more extensive range of well-designed components. Furthermore, with the rise of AI code generation tools, building custom components with these established libraries is often faster and results in cleaner, more maintainable code than relying on the sometimes painfully slow Amplify codegen process. My experience here led to many wasted hours and a decision to avoid them in the future. ## Additional Core Amplify Features Beyond the areas mentioned, Amplify also provides robust solutions for other essential BaaS functionalities: ### Authentication Amplify integrates seamlessly with Amazon Cognito to provide a fully managed user directory. Setting up various authentication flows, including email/password, social logins (Google, Facebook, etc.), and multi-factor authentication (MFA), is relatively straightforward. This is a critical component for most applications, and Amplify handles it quite well, offering both pre-built UI components (though I preferred building my own using the underlying libraries) and fine-grained access control rules tied to user groups and ownership. Its cheap, BUT the docs are rough and it pisses me off that you can’t change the group claims after creation. Migrating cognito user pools is a too much of a hassle. ### APIs (GraphQL and REST) Amplify simplifies the creation of both GraphQL APIs (powered by AWS AppSync) and REST APIs (powered by Amazon API Gateway and AWS Lambda). While I focused on the database challenges with GraphQL’s data modeling, the ability to quickly set up endpoints and connect them to backend logic (often implemented using Lambda functions) is a core strength. Defining resolvers or Lambda handlers allows you to implement custom business logic. ### Functions (AWS Lambda) As I mentioned using Lambda for DynamoDB workarounds, it’s worth noting that Amplify makes it relatively easy to add and manage AWS Lambda functions. These can be triggered by API calls, database events (like DynamoDB streams), or other AWS services, providing a flexible way to add custom server-side logic without managing servers. These features are fundamental to a BaaS, and Amplify’s integration with the broader AWS ecosystem provides a lot of power and scalability potential, even if some aspects require a deeper understanding of the underlying AWS services. ## Conclusion: A Mixed Bag with Potential After three years, my experience with AWS Amplify is a mixed bag. It has been instrumental in allowing me to operate efficiently as a solo developer, providing a unified platform for deploying fullstack applications on AWS. The cost-effectiveness for my current scale is a definite plus, and core features like Authentication and API management are robust. However, the pain points, particularly the awkward fit between Amplify’s data modeling and DynamoDB’s NoSQL nature, and the disappointing UI components, are significant. The slow and uncertain transition to Gen 2 adds another layer of frustration. While the move towards a code-first, CDK-based Gen 2 is promising and might address some of the infrastructure rigidity, the database challenge remains a major consideration. For future projects, the appeal of a SQL-first BaaS like Supabase is strong due to the inherent flexibility of PostgreSQL. Whether I choose Amplify again will heavily depend on the specific project requirements, particularly the database needs, and the maturity and migration story of Amplify Gen 2. It’s a powerful tool, mostly because of the underlying maturity and potency of AWS Services like API Gateway, AppSync, and DynamoDB, but one that requires navigating its quirks and being prepared for workarounds, especially when deviating from its preferred patterns. --- ## Don’t Forget to Ask the Customer Before, During, and After **URL**: https://smiling.dev/blog/dont-forget-to-ask-the-customer-before-during-and-after **Author**: Bryson **Published**: 10/31/2024 **Tags**: Business, Data Science, Software Development, Software Engineering, entrepreneurship **Description**: I just wanna put this out as a PSA that you should be writing and making products that work well AND they solve a end user frustration. Seems stupid to say this until your realize how easy it is for a project at work to carry on 6 month ( or even years!!! ) without getting substantial and valid criticism and feedback. ![](https://miro.medium.com/v2/resize:fit:500/1*mVH46i8ccrRj4SQ1_NXfRA@2x.jpeg) I just wanna put this out as a PSA that you should be writing and making products that work well AND they solve a end user frustration. Seems stupid to say this until your realize how easy it is for a project at work to carry on 6 month ( or even years!!! ) without getting substantial and valid criticism and feedback. Good examples that always get touted as products that fit the market need are Uber, Slack, Tesla, and Zoom. Examples of products that failed because of timing or lack of market value after lots of money and effort thrown at it are New Coke, Google Glass, Microsoft Zune, and Kodak. # **Masters of Entrepreneurship at The University of Notre Dame** For my masters degree, I went to the University of Notre Dame. I was in the [ESTEEM](https://esteem.nd.edu/) program, which had the long name of engineering, science, and technology entrepreneurship excellence masters. well the name may be very long, it was essentially a side step away from the business program. It was amazing for many different reasons, but the one thing that really stuck home was the motto, “**Build things that matter”.** During that masters program, we took on various projects, and one of them required 50 and user interviews. And my case I ended up working with a company that was in healthcare and cyber security, and because of that I had to reach out to 50 people in that space. I was tracking on people on LinkedIn and anyone that would talk to me that had experience as a CTO or CISO for interviews about what their needs and pains were, and how my potential ideas could help. We then would take this and begin to merge the feedback we got with the original idea of the business to make a better and more targeted idea. Eventually we would execute on the idea, either by created the rough business plan and even creating prototypes that could be rolled into further development. I ended up making a really cool idea that the company later that year started to roll with. And I’m super proud of the fact that even though I had no experience with cyber security and no experience with healthcare, I was able to really get something that was grounded and that worked. I’m really appreciative that I learned from a year of experience on how entrepreneurship works. It’s grit and scrappiness to go figure out things that you didn’t know before, but then also to realize that you’re making something that needs to be bought and desired. The best way to do that is to go ask the user do they want what you’re making. # **Corporate Blinders** No one likes to feel like their work is an important. But we also want to feel like we have a secure job that will continue for the next foreseeable future, if we choose to work there. Often times are bigger the company, the more side projects and deep dives into technology there is. I am discovering little by little that the same lessons I learned at the ESTEEM program are still just as applicable to a master student as they are to a software developer. As a developer you have multiple “customers” you have to satisfy: 1. The Customer — they might be on the company, they might be on the other side of the world paying $2 for your business micro transactions. They needs to buy the product and hopefully keep buying. 2. The business — its bottom line is profit. Can your work efforts bring profit? Sadly, we all are cogs in someone’s wheel. 3. Your manager — They need to prove to the business that they are bringing profit and value because of their ideas, management, project, and team. Great managers are a delight to work with/ for. Sucky managers drag you down with their own inability to align incentives and see the bigger picture. 4. Your team — they need to prove that they provide value themselves and keep their job. They are looking for workplace satisfaction and a paycheck just as much as you are. Uplift them as you watch out for your own interest. The “customer” is always a dynamic term depending on perspective. You will have to satisfy/ delight the customer if you want to keep a job, build a team, or sell your product. Keep that in mind. > The customer is always right in matters of taste You might have the know _how_ to build something, but the customer should have a say in the _what_. Remember to look up every once in a while from the corporate requirements and ask yourself, how’s the actually helping the end customer? # **Okay, but how?** I guess that’s kind of the beauty entrepreneurship is there’s no one right way to do anything. I don’t know your job, I don’t know your project, and I don’t know what your workplace environment looks like. Hear the basics: 1. Ask yourself how can I deliver value? If that value is completing a bunch of Jira tickets, that’s great. If you’re starting a Greenfield project, be sure to ask the customer where they actually want to use this. And also ask them. 2. Beware of the XY problem. Sometimes customers say they want one thing, but they really want another. This takes a little bit of experience to see, but you’ll notice it when in the beginning of projects you don’t have something totally firm down or the customers are describing a problem they think they have. One of the most famous characterizations of this is when Ford said, “If I had asked people what they wanted, they would have said faster horses.” 3. In the beginning, iterate quickly and be unafraid to change or start over. Make a low fidelity demo (think figma, balsamiq, ugly react app, scraps of paper with labels) that you can make an iterate quickly over. Ideally, if you can do it within a day and get feedback from your end-user in a day, you doing something right. 4. Get customer and stakeholder buy in. You can have the best idea in the world that solve someone’s problem, but if you can’t explain the idea, well, you won’t get far. 5. What venture funds look for is the Right Team, the right Market, and the right Timing. managers will look for the same things, whether they know it or not. Do you have the right skill set to get things done, are you trying to address a problem that is worthwhile, and do you have enough time and timing to get it done? # **Conclusion** I hope some of these ideas help you in your next engineering project. I’ve seen firsthand how thinking like an entrepreneur can elevate your career and bring energy back into your work. Heck, you might even want to go be a solo entrepreneur after this ha ha --- ## What is Pydantic and how can it help your next python project? **URL**: https://smiling.dev/blog/what-is-pydantic-and-how-can-it-help-your-next-python-project **Author**: Bryson **Published**: 10/6/2024 **Tags**: Software Development, Python, Software Engineering **Description**: For anyone that has been either living under a programming rock or honestly has stuff to do outside of python development, you may not have heard about Pydantic. ![Spidy knows a good validation library will help protect your local neighborhood](../images/what-is-pydantic-and-how-can-it-help-your-next-python-project.webp) Of course. Here is the text formatted in markdown. For anyone that has been either living under a programming rock or honestly has stuff to do outside of python development, you may not have heard about **Pydantic**. It's a package for data validation, object serialization and de-serialization, and class management. In my recent article, *Stop using Python like it was 15 years ago*, I mentioned you should be using Pydantic models. So now I’m going to show you what I mean. Also, just found out a little while ago that the definition of *pedantic* is: > “someone who annoys others by correcting small errors, caring too much about minor details, or emphasizing their own expertise especially in some narrow or boring subject matter.” So this is very fitting for a medium tech article! haha ## What do I use it for? * Parsing incoming **data** * Passing around **settings**, **data**, and other information over **args** * Ensuring that **types** are what I expect at runtime. *** ## 1. Parsing Incoming Data Something I have done a few times is make a lambda function in python that does some kind of data pipeline or transformation. A lambda function will get an incoming payload of data (usually from API Gateway or SQS) and then process it. ([Link to AWS Docs](https://docs.aws.amazon.com/lambda/latest/dg/services-apigateway-tutorial.html#services-apigateway-tutorial-function)) Let's say you are making a PayPal transaction processing lambda function behind an API Gateway. You need to parse the incoming request, call Paypal `POST /order` to create a new order, then return back the order ID to the frontend so that the client can accept the transaction. The **brittle** way to do this is: ```python def handler(event, context): if event and "user_id" in event and "db_order" in event: create_order(event.user_id, event.db_order, event) else: return "bad request" ``` It's brittle because you are not **validating** that `user_id` and `db_order` are UUID or the form you are expecting. This is a simple example, but if you are trying to parse in many args from the event, it's easy to mess it up. Also it sucks to read. Try this: define a Pydantic model, then pass the dict into it. If it passes, then you continue. If it fails, then you assume it's a bad request. ```python from uuid import UUID from pydantic import BaseModel, ValidationError class CreateOrderRequest(BaseModel): user_id = UUID db_order = UUID discount_code = str | None def handler(event, context) -> str: try: request = CreateOrderRequest(**event) create_order(request) except ValidationError: logger.exception(f"Failed to parse event: {event}") return "Bad request" ``` It's much more **clear** about **what data you are expecting**, **what is optional**, and also the Pydantic library will check the **types of these parameters** automatically! *** ## 2. Passing around settings, data, and other information over args I like this one because my future self appreciates the reduced maintenance. Why? Because coming back to code that is 6 months or older is tricky because you forgot **why you did certain things**. You forget the **assumptions** the code is making. So make it explicit! Let's assume we are making an image processing pipeline. The code is going to download a requested image (from AWS S3), do a magic AI transformation on it, then save it back in another location. Here is the **brittle way**: ```python def image_pipeline(object_key: str): default_d_bucket = os.environ["download_bucket"] default_u_bucket = os.environ["upload_bucket"] # might cause index error path = download_image(object_key, default_d_bucket) do_ai_magic_transform(path) upload_image(object_key, path, default_u_bucket) def download_image(object_key: str, bucket: str): boto3.client('s3').download_file(bucket, object_key, f"/tmp/{object_key}") return f"/tmp/{object_key}" def do_ai_magic_transform(path: str): # Placeholder for AI transformation pass def upload_image(object_key: str, path: str, bucket: str): if os.environ.get("is_special_object"): bucket = "special_bucket_name" else: logger.info("uploading to default location") with open(path, "rb") as file: boto3.client('s3').put_object(Bucket=bucket, Key=object_key, Body=file) if __name__ == "__main__": image_pipeline("example.jpg") ``` This is only 3 small functions but it takes a few moments to understand what is going on, what environment variables are needed, why do we need a special upload bucket for special occasions, etc. Now all information is centralized and you can more **easily understand the variants in this pipeline**. Also, the model type checks the information so you are not accidentally passing around a `None` at runtime. ```python from pydantic import BaseModel, Field logger = logging.getLogger(__name__) class Config(BaseModel): object_key: str download_bucket: str = Field(..., env="download_bucket") upload_bucket: str = Field(..., env="upload_bucket") temp_dir: str = Field(default="/tmp") is_special_object: bool = Field(default=False, env="is_special_object") special_bucket_name: str = Field(default="special_bucket_name") path: str | None= None def init_config(object_key: str) -> Config: config = Config(object_key=object_key) config.path = os.path.join(config.temp_dir, config.object_key) return config def image_pipeline(config: Config): download_image(config) do_ai_magic_transform(config) upload_image(config) def download_image(config: Config): boto3.client('s3').download_file(config.download_bucket, config.object_key, config.path) def do_ai_magic_transform(config: Config): # Placeholder for AI transformation logger.info(f"Applying AI transformation to {config.path}") def upload_image(config: Config): bucket = config.special_bucket_name if config.is_special_object else config.upload_bucket logger.info(f"Uploading to bucket: {bucket}") with open(config.path, "rb") as file: boto3.client('s3').put_object(Bucket=bucket, Key=config.object_key, Body=file) if __name__ == "__main__": config = init_config("example.jpg") image_pipeline(config) ``` You can even avoid having to pass around the config to each function with just a function called `get_config()` which would return the config object. *** ## 3. Ensure the types are what I think at runtime This one is the worst because it's why python, I think, can get a bad rap with programmers that are used to Java or C++. More than I want to admit, I’ve written some python code that seemed fine, I probably should have written more unit tests, but it worked fine. So I push it to deployment in the dev environment and it breaks something. I missed a `None` that was returned, or an API that I was calling returning a nested nested dictionary when I was expecting just the list of strings. The list can go on and on, but it comes down to that one of Python's strengths is also a weakness: its **type flexibility**. So for that reason, when you have data coming from an external source, just throw it in a Pydantic model. If it passes, great. If not, then you hopefully can fix it more quickly. Stay safe out there! Check your types, don’t code while driving, and enjoy your weekends by not working! --- ## Stop making your python projects like it was 15 years ago… **URL**: https://smiling.dev/blog/stop-making-your-python-projects-like-it-was-15-years-ago- **Author**: Bryson **Published**: 9/28/2024 **Tags**: Software Development, Python, Software Engineering **Description**: I have a few things I’ve seen across companies and projects that I’ve seen working with Python that are annoying, hard to maintain, and are antiquated. Lets jump right in ![A sad programmer thinking about the time wasted with python project management and debugging](../images/stop-making-your-python-projects-like-it-was-15-years-ago-125436b470a5.png) I have a few things I’ve seen across companies and projects that I’ve seen working with Python that are annoying, hard to maintain, and are antiquated. Lets jump right in: ## Use PyProject.toml instead of requirements.txt Don’t do it. Stop putting in your README.md > install dependencies with pip install -r requirements.txt Can pip do it? Yes. Should you? No. requirements.txt is a accidental standard. Just use `pyproject.toml`. It makes sense, pip understands it, you can define dev and groups of dependencies in it so you don’t have to start using “dev-requirements.txt” and a slew of cicd pipeline rules. See [this python subreddit conversation](https://www.reddit.com/r/learnpython/comments/1bmxe6i/whats_the_difference_between_pyprojecttoml) ## Use a python version and project manager like Poetry or UV I’m partial to UV, but I like Poetry too. They both will read the pyproject.toml file, handle dependencies, building, running applications in sandboxes, managing version of python and more. It will take you an hour to read about these tools and it will save you head ache. At a bare minimum, use a virtual environment like `venv` to save yourself some global dependency headaches. Also `UV` will install dependencies way faster than pip. ## Use Type hints Type hints are great for readability and helping linters like ruff and mypy know what the hell is happening in your code. Your coworkers will like your code better too. This is hard to understand when I have to pick up your code 2 years later: ```py def do_something(data): for x in data: for y in data[x]: transmute(data[x][y]) ``` Make it easier: ```py def alter_map(data: Map) -> None: """ Alters data in a 2d map. Alters the object refernced directly """ for x in map: for y in map[x]: transmute(map[x][y]) ``` Add type hints to all parameters in functions, return types, etc. ## Add a Raises section to your function docstrings This might be controversial because people will say that it doesn’t stop bad code or enforce anything, but it still helps to maintain code. In your docstrings, add a Raises and the possible exceptions that can be raised from this function to minimize surprises: Now this is helpful! ```py from typing import Dict, Any, optional def query_db(query: SqlQuery) -> Optional[Dict[str, Any]: """Calls the database Args: ... Raises: ClientException: you goofed up TimeoutException: db took too long ConnectionException: something is wrong with your network Returns: The response from the query, if any. """ ``` ## Use Pydantic Models to pass around data and parameters over dicts or numerous function parameters I’m tired of code I see that has functions that essentially prop drill with pieces of data that don’t change. This a data processing pipeline that has a bunch of environment variables that get looked up once and never change. Just define a pydantic class and boom, you have error handling, type checking, and can pass that around as a single argument to functions. ## Use a linter and formatter like Ruff Javascript projects have prettier and its a near standard at this point. In Python, use ruff (which copies Black’s formatting standard) and its fast. it also will lint common issues that you might have missed. this linting is not as in depth as a static analysis tool like mypy, but sure does help. While you’re add it, add a few of these opt-in rules into your pyproject.toml to make your code base even better: ```toml [tool.ruff] target-version = "py12" [tool.ruff.lint] extend-select: [ 'D', #pydocstyle 'E', 'W', # pycodestyle 'F', #pyflakes 'I', # sort imports 'UP', #pyupgrade "RUF", # ruff dev's own rules "SIM", # pyflakes simplicity "C90", # more complexity rules ] [tool.ruff.lint.pydocstyle] convention = "google" [tool.ruff.lint.isort] combine-as-imports = true split-on-trailing-commas = false ``` There are many things you can add here. But is al a carte. ## Use Pytest over unittest Where is makes sense to convert, use pytest over unittest. The pytest fixtures are very helpful and composable. ## Hot Takes If You Can Manage To Sneak This Into Your Projects - Use orjson instead of the built in json library - Always use f strings instead of string concatonation or .format or %s formatted strings - Use pathlib instead of os.path - Use [click](https://click.palletsprojects.com/en/stable/) or [typer](https://typer.tiangolo.com/) instead of argparse or heaven forbid, sys.argv - Upgrade to python 3.8+ *Thank you for coming to my soapbox Ted Talk.* ---