Minimal changes

.github/workflows/docs.yml

@@ -35,7 +35,8 @@ jobs:
             - id: install-deps-and-build
               name: Install dependencies and test
               run: |
-                pip install -r requirements.txt
+                pip install -e .
+                pip install -e .[doc]
                 mkdocs build --site-dir build
 
             - name: Setup Pages
@@ -52,3 +53,10 @@ jobs:
             - name: Deploy to GitHub Pages
               id: deployment
               uses: actions/deploy-pages@v2
+
+            - name: Upload PDF
+              id: upload-pdf
+              uses: actions/upload-artifact@v3
+              with:
+                name: document.pdf
+                path: site/pdf/document.pdf

.gitignore

@@ -158,3 +158,6 @@ cython_debug/
 #  and can be added to the global gitignore or merged into this file.  For a more nuclear
 #  option (not recommended) you can uncomment the following to ignore the entire idea folder.
 #.idea/
+
+
+*.db

Caddyfile

@@ -0,0 +1,8 @@
+:80 {
+    handle_path /api/v1/* {
+        reverse_proxy localhost:8000
+    }
+    file_server {
+        root src/retailtwin/api/static
+    }
+}

README.md

@@ -1,2 +1,45 @@
 # retailtwin
-A small digital twin of a supermarket chain to showcase some relevant aspects of transactional systems.
+
+* [documentation](https://psychic-broccoli-r4ome5j.pages.github.io/)
+
+Implementation of a digital twin of a retail corporation that operates a chain of grocery stores will be described. The implementation is of course limited but comprehensive enough to get some key insights about why corporate IT looks the way it looks. If anyone intends to create antifragile data systems it's important to study how fragile systems come to be on the first place.
+
+## Bootstrap the database
+
+First create a postgresql database.
+
+```bash
+createdb -h localhost -U postgresuser retail
+```
+
+In this case we decided to call the database `retail` and we created it in the same computer we will be running the digital twin. Then sync the data models in the freshly created database with:
+
+```bash
+retailtwin init postgresql://postgresuser:password@localhost/retail
+```
+
+Then we can populate the database with dummy data with the `bootstrap` subcommand:
+
+```bash
+retailtwin bootstrap postgresql://postgresuser:password@localhost/retail
+```
+
+Finally we can create all the necessary functions, procedures, and triggers with
+
+```bash
+retailtwin sync postgresql://postgresuser:password@localhost/retail
+```
+
+## Terminals
+
+There are currently three available terminals to operate with the digital twin:
+
+### Stocking terminal
+
+```bash
+stock [DB_URI] [Store location]
+```
+
+```bash
+stock postgresql://postgresuser:password@localhost/retail 1
+```

docs/api.md

@@ -0,0 +1,3 @@
+# API
+
+API stands for Application Programming Interface. 

docs/applications.md

@@ -0,0 +1,7 @@
+# Applications and value
+
+!!! example
+
+    While in a pitch, one of the clients said that she didn't want the call center agents to look at multiple screens. If we developed anything we had to integrate it into their current tool. That meant that we couldn't develop exactly what we thougth could bring more value, but we had to study what could be integrated with the current tool instead. I really wanted to challenge that. Her hypothesis was that more complexity made call center agents less productive, but I've seen examples of the opposite. While tuning a similar application in a previous client, I did the same challenge to the applicaiton developers. Maybe we were showing too much things in the screen. But once we put the applicaiton in front of call center agents, they asked to see even more informaition! 
+    
+    Assumptions have to be tested, and it's surprising to me how consultants challenge client's assumptions on how they run their business, but we decide not to contewst important choices about usability and design. That client was more than willing to sacrifice tons of functionality, and maybe millions, due to an hypothesis that could perfectly be wrong.

docs/automation.md

@@ -16,4 +16,9 @@ For each automation you should decide the execution time, with many possible cho
 
 # Triggers to act on a given condition
 
-Let's open the `stock` terminal run a little experiment.
+Let's open the `stock` terminal run a little experiment.
+
+# If it works, don't touch it
+
+Creating robust automations is key for the operations of any corporation. Automations tend to become systemic, and if somethigs works as expected, there's a strong motivation to touch it as little as possible. Even nowadays it's common to find out that the automations dealing with payments are still implemented with COBOL, and in that case, the process to migrate from COBOL to other "enterprise" language like Java, takes half a decade to complete.
+

docs/buyvsbuild.md

@@ -1,5 +1,7 @@
 # Buy vs Build
 
+Modularization of enterprise software components. This makes running businesses more sustainable.
+
 ## Bulid vs buy an off-the-shelf solution.
 
 Bulding enterprise data systems is hard and expensive. This is why many companies decide to purchase an ERP like SAP, which is able to handle financials, purchasing, stock, customer relationships, reporting... This seems to be the safest choice by far
@@ -18,7 +20,7 @@ But appearances can be deceiving.
 * Proprietary software is always bundled with some additional vendor lock, like supporting a small subset of proprietary storage systems.
 * Buying software implies vendoring knowledge. If there's a critical issue with one of these components and the vendor is not able to provide support, or it bankrupts and disappears, the engineers within the corporation won't be able to fix the issue no matter how smart they are.
 
-The most common scenario is to run a mix of applications: mission-critical operations are run by custom, in-house-developed applications maintained by the IT department, while other less critical operations like Marketing use third party tools. I've encountered many corporations that run most of their operations on custom-built software, but they decided to buy a proprietary CRM like Salesforce to support their Marketing and Sales departents. Most projects executed by IT deparments are related to data integrations between two existing tools, or between existing and some new tool the leadership decided to purchase.
+The most common scenario is to run a mix of applications: mission-critical operations are run by custom, in-house-developed applications maintained by the IT department or functional areas, while other less critical operations like Marketing use third party tools. I've encountered many corporations that run most of their operations on custom-built software, but they decided to buy a proprietary CRM like Salesforce to support their Marketing and Sales departents. Most projects executed by IT deparments are related to data integrations between two existing tools, or between existing and some new tool the leadership decided to purchase.
 
 ## Bespoke software and the false buy vs build dichotomy.
 

docs/index.md

@@ -0,0 +1,3 @@
+# Navigating IT
+
+This document is an essay that supports a course

docs/organization.md

@@ -1,2 +1,117 @@
 # People in the data ecosystem
 
+The human dimension plays an important role in any system. Any individual has motivations and fears that have to be taken into account when planning an implementation. 
+
+In some organizations there's some degree of antagonism between the functional areas and IT. While the former mostly try to move the business forward implementing new or improved processes, and generating value, the latter is more conservative because there's a wider set of goals to achieve apart from supporting business objectives:
+
+1. Enhancing Security and Compliance. That service may take a couple more weeks to get online because there's a penetration test to run.
+2. Managing their own budget. That Spark cluster the data team requires is way over the year's budget so there will be some more waiting ahead for the project's kick-off.
+3. Providing End-User Support. Let's restore a backup for the junior developer that accidentally deleted all the tables of the development database.
+4. Managing IT Operations. This Saturday one of the senior engineers will truncate some tables and rebuild the indices in the central Oracle database because the average measured transaction latency went past the threshold last week.
+7. Developing IT Talent. It seems GenAI is a thing now, let's give the Architecture team a couple weeks to understand how we can run LLM in our production systems without bankrupting the whole company.
+8. Building IT Partnerships. We're at 100% capacity just keeping the ligths on, maybe we need to outsource network management for business users so let's call zscaler.
+9. Ensuring Business Continuity. Let's make sure that every single server, laptop, cell phone, and database within the company is backed up at least once a week.
+
+On some corporations the IT team is also responsible, not only of the operation of in-house applications and automations, but their design and implementation too. When that is the case, the IT department is the most relevant transversal team within the company.
+
+The top layers of the IT organization will probably be business focused, and their incentives will be aligned with other functional areas. However, all staff below IT directors will not be motivated by the financial success of the company. You may have the buy-in from the CTO, while facing major setbacks with the director of IT operations, and this requires some additional thinking.
+
+## Value Vs Risk
+
+In essence, *functional areas try to maximize value, while the IT department tries to minimize risks*. The IT managers know that they will be able to capitalize a limited share of each success, but they will be made responsible for incidents. Any plan that may incur into additional risks will be scrutinized the sooner or later because there's no way a new digital product or initiative gets implemented without the involvement of IT.
+
+This is the framework I use to bring some structure to this tension:
+
+1. Ideas are about results, not about implementations
+2. Value lies in without results
+3. Risks are inherent to implementation
+4. There are no results without implementation
+5. There is no value without risks
+
+Project execution improves where you think about engineers and managers in the IT department as someone who can *reality check your ideas*. If you're a consultant that mostly works on the "ideas" side of the equation, they will force you to think about an implementation, and then they will challenge it. Their goal is to make the final result better and be helpful providing a new point of view. If you think of them as a blocker to get from idea to results and ultimately value, you're following the wrong approach. Your goal is to have foundational knowledge to understand their language, digest their ideas, improve the final result, and maximize the value.
+
+This text provides the tools you need to engage with discussions with members of the IT team. When a data owner says that the KPI you need is not in the data warehouse, but on a transactional system, and there's no ETL that currently in place for that, so you have to create a ticket to the data engineering team and join the discussion on the next sprint planning... I can assure that person is more than willing to help you, laying out exactly what you need, and your necessary next steps. The answer could have been that the transactional system is legacy, it can't sustain more load, and they're not planning to extract any more data from it. Again, that person is not trying to sabotage your project, it's trying to prevent a major meltdown within the company.
+
+!!! note
+
+    Are only results valuable, or is the implementation valuable too? This is almost a philosophical question. My take is that you can perfectly argue that implementations have inherent value, but it's very hard to measure. Once you follow the "implementations have value" way you can't be picky about which bits of the implementation are more valuable than others unless you explicitly measure that. I once had a discussion with a member of the Lighthouse (an internal BCG product that provides a self-service interface for third party data) team during an internal meeting. I was representing the Atlas team, which builds platform components and development ecosystems. He argued that platform has no value, since it's just an enabler that's far from the results. I immediately responded that data is an enabler too: if platform has no value, neither has data. Data are closer to the results, but they're are not results either.
+
+
+## Implementation, ownership, and budget.
+
+Risk management involves
+
+* Responsibility
+* Alerts
+* Minimizing impact
+
+It's very common to forget about IT costs when estimating the value of a case.
+
+It's very common to ignore about information security, and constraints with personal information, when pushing for value.
+
+IT may come with a multi-million euro request to run your project. This is why many corporations are reluctant to build capabilities that run on razor-thin margins. Plans that gent implemented need an early estimation of.
+
+The most pressing risk is the one of oversimplification. Business leaders tend to focus on what's possible, and underestimate the difficulties hidden on the implementation details. I call this the Elon Musk Syndrome.
+
+!!! note
+
+    Elon Musk has systematically made claims to customers and investors that could not meet afterwards. From systematically failing to predict the number of Tesla cars built each year, to the development of [Optimus](https://en.wikipedia.org/wiki/Optimus_(robot)). Fortunately for Musk, he's not running a traditional corporation, that would get systematically take hit on their sales for these unmet claims, but a religion.
+
+## Teaming with IT
+
+Fully functional BCG case teams will staff engineers from X and architects from PLA that will deal with all the IT-related details, but sometimes teams are understaffed, or the budget is insuffcient to allocate the necessary profiles. Sometimes a core senior associate has to lead the relationship with IT.
+
+* Involve them as soon as possible
+* Let them contribute
+* Give them details about the goals, and how you plan to achieve them
+* Help them capitalize the win
+
+The most usual reason why IT may not be willing to help is because they don't have the capacity to run more things. If they're just delaying things, or not helping you figure out which is the best way to go from idea to implementation, probably it's because they're not able to fit your needs in their planning. This is why working with them from the beginning of the project is relevant. You may team with them to:
+
+1. Get more ownership from functional areas.
+2. Help them plan the budget.
+3. Echo their staffing needs. In the end the goal of every department is to grow in size and importance.
+4. Praise their work
+
+## Antipatterns
+
+One consequence of the tension between value and ownership is the existence of [Palantir](https://www.palantir.com/). The value proposion of Palantir is that's impossible to capture value from data analytics and applications when the infrastructure is run by IT departments. According to their pitch, it just takes too long to get anything done, and they can't be trusted to build a valid development ecosystem suitable for advanced analytics. The only way around that issue is to create a set of data integrations to push data to the cloud, and pay a monthly subscription for their hosted analytics solution.
+
+This just trades a short-term issue into a long term risk: vendor lock.
+
+It also neglects how interwouned IT is with the rest of the organization. Assume that the CFO needs to execute aggressive cost cuts, takes a look at the different providers, and sees €1.5M/year for Palantir in the Marketing division's budget. But Palantir is a platform for data! How come the director of IT never signed off that spending, or at least was asked to provide an alternative? And while other departments are laying off personnel, and shutting down non-critical applications, a significant portion of the Marketing funds are spent in the most aggressive vendor-locked asset that there is.
+
+Thinking about the entire value chain, and the complete timeline, including short and long term consequences, is something that the management consultants (that mostly deal with strategy) should know what to do.
+
+## With cloud, IT may be split.
+
+In some organizations the IT assets may be split. Historically, IT assets used to be hosted as part of the organization's own infrastructure, and some of the budget was spent purchasing servers and storage cabinets. 
+
+Cloud has raised the level to access.
+
+## Dysfunctional IT departments
+
+There are many reasons why an IT department becames dysfunctional, but the main one comes from the tendency of turning IT into a kitchen sink of ownership. Healthy organizations try to align responsabilities with incentives, but it's relatively uncommon. Assume a b2b business that is growing fast, and the Marketing division needs a new CRM to be able to handle all the new customers. In a healthy organization IT will be involved in the planning stage as part of a joint team with all the stakeholders. Involving IT may have unintended consequences, like discarding the Marketing team's preferred software because the technological stack is imcompatible with the current and future IT strategy. In an unhealthy organization, the CMO will tell the CTO that software X must available, and the request will be waterfalled to the IT department that will need to figre out how much it costs, and find a way to integrate it, operate it, evolve it...
+
+If this situaition extends for a long time, the IT department isolates as a mechanism of self-defense, and gravitates towards team members with a strong tribal culture. Since IT is a team that only deals with ownership, risks, and responsbilities, then they will make their work more opaque to others and decide what to do, and the best way to do it. In the long term isolation turns into some sort of hostility.
+
+!!! tip
+
+    There's a rule of thumb to estimate the friendliness of an IT department. Friendliness tends to be proportional to the presence of women.
+
+If none of theat works, call the experts. There are many seasoned engineers at BCG who can help you execute in the harshest environments, even with hostile IT departments. Sometimes the final decision is to go nuclear and let BCG host the solution while requesting the minimal help from IT. This makes the project significantly more expensive, but it's often the only way to reach the goals that were agreed with the client.
+
+## People
+
+Each organization implements their IT capabilities in a slightly different way
+
+* CTO. (Chief Transformation Officer, Chief Transformation and Product Officer...) Value based, process centered, not really into the technical details. May be in charge of IT (some companies don't have a CIO).
+* CIO. Oversees everything related with IT, including systems, procurement, strategy... May report to CTO.
+* CISO. Also takes care of information security (devices, network, policies, access)... May report to CIO or CTO.
+* Director of Systems. Owns the IT's budget.
+* Director of Architecture / IT. Owns new assets.
+* Director of Innovation. Owns new applications. May own part of budget.
+* Director of Data / Engineering. Owns data health and availablity.
+* Director of Data Science / Analytics. Owns data insights.
+* Director of *division*.
+* Cloud architects

docs/pragmatism.md

@@ -1,6 +1,6 @@
-# Introduction
+# Pragmatism
 
-This long chapter intends to answer why all the techonologies and techiniques covered in this text are seldom used in an enterprise environment. There has been a substantial effort to introduce the techniques to implement data management pipelines in the most efficient way. It's humbling to find out that what we see in clients points in the opposite direction:
+It's the norm in enterprise IT divisions to disregard "modern" engineering tools and techniques:
 
 1. The use of open-source DBMS tends to be marginal, and Oracle still dominates the market.
 2. Automation is managed by old-school enterprise management tools like Control-M, or

docs/terminals.md

@@ -77,9 +77,9 @@ q> 566807566244
 └──────────────┴───────┴────────────────┴─────────┴────────────────────────────┴────────────────────────────┴──────────┘
 ```
 
-## If it's smart it's vulnerable
+It's frequent to assume that the most usual way to interact with enterprise data nawadays is with modern web-based interfaces. But there many old-school terminals still around. Point of Sales terminals tend to be very basic as well, with displays only capable of showing a handful of characters, and a button for each command. Being this a digital twin, with complete freedom to implement anything we want, building a command-line terminal is also a way of making a point.
 
-It's frequent to assume that CLI terminals are outdated, and more modern web-based user interfaces are the most common. But there many old-school terminals still around. Point of Sales terminals tend to be very basic as well, with displays only capable of showing a handful of characters, and a button for each command. Being this a digital twin, with complete freedom to implement anything we want, building a dumb terminal is important to introduce the following point:
+## If it's smart it's vulnerable.
 
 The most important constraint when designing enterprise data systems is information security, and the dumber the terminal, the more secure it is. PoS tend to be dumb because there's money inside. One key concept in information security is the *attack surface* of a system. A console with no graphical interface and a handful of commands connected to a database is inherently more secure than a web interface that needs a browser, a http connection, a web server, and a database. I can't recommend enough the book [If it's smart it's vulnerable](https://www.ifitssmartitsvulnerable.com/) by the veteran information security researcher Mikko Hypponen. Maybe that $200 cloud-connected PoS with a fancy screen from Alibaba is the door someone exploits to start a ransomware attack, or that simple web terminal that the cheapest bidder implemented is vulnerable to SQL injection.
 
@@ -87,12 +87,32 @@ The most important constraint when designing enterprise data systems is informat
 
 [From XKCD](https://xkcd.com/327/)
 
-CLI Terminals also run everywhere, and require almost no support from the operative system. Here's the Windows command prompt running the `stock` terminal application.
+## If it's complex is expensive.
+
+Mankind has been spoiled by intuitive and ergonomic user interfaces since the iPhone appeared, but mankind also landed on the moon using using a computer with the most spartan user interface ever.
+
+!!! example "Control panel of the Apollo Guidance Computer"
+
+    ![Apollo.jpg](https://upload.wikimedia.org/wikipedia/commons/b/bd/Apollo_display_and_keyboard_unit_%28DSKY%29_used_on_F-8_DFBW_DVIDS683588.jpg)
+
+    Here's a [working simulator](https://svtsim.com/moonjs/agc.html) where you can follow the full launch sequence that the crew of the Apollo spaceship had to introduce on the computer.
+
+CLI Terminals are robust, run everywhere, and require almost no support from the operative system. Here's the Windows command prompt running the `stock` terminal application.
 
 ![terminal.png](img/terminal.png)
 
 There's a 99% chance that the future Windows version released in 2033 is still able to run this application. That may not be valid for a web-based application developed with today's technologies. The most popular browser technology in corporate clients ten years ago was still Internet Explorer, and web applications had to implement support for it.
 
+!!! example
+
+    The Airbus A320 civil aircraft was developed in the eighties. The Multipurpose Control and Display Unit (MCDU) is a panel that the flight crew use to interact with the onboard computer. Together with the autopilot, it's one of the components of the aircrafts that the crew spends most time interacting with.
+
+    ![mcdu.png](img/A320cockpit.png)
+
+    It took more than 20 years to move from a simple keyboard and a 5-inch screen to a trackball and keyboard when the A380 was developed. The most modern aircraft by Airbus, the A350, features the Keyboard and Cursor Control Unit (KCCU) with a QWERTY keyboard and a pointer that they can move around the panels in the cockpit. It's more modern, intuitive, enjoyable, and less error-prone.
+
+    New Airbus A320 still get a MCDU. There's very little motivation to upgrade a design that works: the Airbus A320 is the [highest-selling airliner](https://en.wikipedia.org/wiki/Airbus_A320_family#:~:text=As%20of%20August%202023%2C%20a,since%20its%20entry%20into%20service.), and there are tens of thousands of crews that already know how to use MCCU. In addition, retrofitting a KCCU into the A320 design may cost to Airbus almost as much as designing a new plane from scratch.
+
 ## API-based web applications
 
 Some terminals, like PoS, run on specific hardware with a dedicated display and user interface. CLI terminals' display is the operative system's console. Web applications' display is a browser, which is today almost as capable as an operative system. The entire Microsoft Office suite can now run on a browser.

mkdocs.yml

@@ -1,16 +1,22 @@
-site_name: My Docs
+site_name: Navigating IT
+site_author: Guillem Borrell PhD
+site_url: https://github.com/bcgx-gp-atlasCommunity/retailtwin
+copyright: © Guillem Borrell Nogueras, The Boston Consulting Group. All rights reserved.
 repo_url: https://github.com/bcgx-gp-atlasCommunity/retailtwin
 edit_uri: edit/main/docs/
 
 nav:
-  - "Introduction": "introduction.md"
+  - "Introduction": "index.md"
   - "People in the data ecosystem": "organization.md"
+  - "Pragmatism in Enterprise Software": "pragmatism.md"
   - "How data is stored": "data.md"
   - "Terminals to access data": "terminals.md"
   - "How automation is implemented": "automation.md"
-  - "Buy vs Build": "buyvsbuild.md"
   - "The Data Warehouse": "dw.md"
   - "The Data Lake": "dl.md"
+  - "API": "api.md"
+  - "Applications and value": "applications.md"
+  - "Buy vs Build": "buyvsbuild.md"
 theme:
   name: "material"
   palette:
@@ -45,7 +51,8 @@ theme:
 
 plugins:
   - search
-#  - with-pdf
+  - with-pdf:
+      cover_subtitle: A course for core consultants and data scientists.
 
 markdown_extensions:
   - md_in_html

pyproject.toml

@@ -11,16 +11,21 @@ dependencies = [
   "duckdb",
   "pydantic",
   "typer",
+  "rich",
   "pyyaml",
   "pydantic-settings",
   "polars",
+  "pandas",
   "pyarrow",
   "sqlalchemy[asyncio] > 2.0.13",
   "adbc-driver-postgresql",
   "adbc-driver-sqlite",
   "prompt_toolkit",
   "asyncpg",
-  "pydantic-settings"
+  "psycopg2-binary",
+  "pydantic-settings",
+  "fastapi",
+  "uvicorn"
 ]
 
 

src/retailtwin/api/README.md

@@ -3,7 +3,7 @@
 Run the backend with
 
 ```bash
-uvicorn retailtwin.api.main:app
+DB_URI=postgresql+asyncpg://user:password@server/dbname uvicorn retailtwin.api.main:app
 ```
 
 Use Caddy to reverse proxy and the following Caddyfile. Paths are static so you have to run caddy from the root of the package
@@ -27,4 +27,4 @@ caddy run -c Caddyfile
 
 You should be able to browse the application at http://127.0.0.1, and reach the api docs at http://127.0.0.1/api/v1/docs
 
-Caddy can be installed in Windows with Chocolatey
+Caddy can be installed in with Chocolatey on Windows, and with Homebrew on mac.

src/retailtwin/bootstrap.py

@@ -24,7 +24,7 @@ from retailtwin.models import (
 )
 
 # Some configuration parameters.
-PACKAGE_ROOT = Path(retailtwin.__file__).parent / "retail"
+PACKAGE_ROOT = Path(retailtwin.__file__).parent
 PRODUCT_LIST_FILE = "data/products.csv"
 DISCOUNT_LIST_FILE = "data/discounts.csv"
 RANDOM_PEOPLE_FILE = "data/random_people.csv"

src/retailtwin/cli/pos/main.py

@@ -4,7 +4,7 @@ Terminal that mimics what a Point of Sale may operate like.
 import typer
 import polars as pl
 from retailtwin.cli.db import query_local_batches
-from retailtwin.cli.pos.models import Base, Sync, Direction, Cart, Item
+from retailtwin.cli.pos.models import Base, Sync, Direction, Cart, Item, ItemOnCart
 from retailtwin.utils import db_uri_from_session
 from datetime import datetime
 
@@ -23,6 +23,11 @@ HELP = """
 # Retail twin POS
 
 This is a simple terminal that simulates a dumb PoS with a single-line screen.
+
+* **n**: Opens a new cart
+* **l**: Lists carts
+* **r**: Refreshes with the central database
+* **q**: Quits the terminal
 """
 
 CACHE_FILE_URL = "sqlite:///pos.db"
@@ -50,16 +55,17 @@ def sync(session: Session, db_remote_uri: str, location: int):
         inventory.select(
             [
                 pl.col("upc"),
-                pl.col("price"),
-                pl.col("discount_definition").alias("discount"),
+                pl.col("unitprice"),
+                pl.col("discount_definition"),
+                pl.col("discount_name"),
                 pl.col("received"),
                 pl.col("received").max().over("upc").alias("max_received"),
             ]
         )
         .filter(pl.col("received") == pl.col("max_received"))
-        .select([pl.col("upc"), pl.col("price"), pl.col("discount")])
-        .with_columns(pl.col("price").str.strip().cast(pl.Float32) * 100)
-        .with_columns(pl.col("price").cast(pl.Int32))
+        .select([pl.col("upc"), pl.col("unitprice"), pl.col("discount")])
+        .with_columns(pl.col("unitprice").str.strip().cast(pl.Float32) * 100)
+        .with_columns(pl.col("unitprice").cast(pl.Int32))
     )
     items.write_database(
         "items", db_local_uri, if_exists="replace", engine="sqlalchemy"
@@ -125,12 +131,15 @@ def sync(session: Session, db_remote_uri: str, location: int):
         session.commit()
 
 
+def checkout(session: Session, cart_id: int):
+    return None
+
+
 def load_items(session: Session):
     return [str(it) for it in session.scalars(select(Item.upc)).all()]
 
 
 def handle_command(command: str, cart_id: int, item_id: int, **kwargs):
-    print(command, cart_id, item_id)
     session = kwargs["session"]
 
     if command == "n":  # Start a new cart
@@ -140,13 +149,32 @@ def handle_command(command: str, cart_id: int, item_id: int, **kwargs):
 
         return cart.id, "", ""
 
+    elif command == "l":  # List carts
+        carts = pl.read_database(
+            "select * from carts", db_uri_from_session(session), engine="adbc"
+        )
+        print(carts)
+        return "", "", ""
+
+    # Handle pushing items to the cart
+    elif cart_id is not None and item_id:
+        session.add(ItemOnCart(cart=cart_id, upc=item_id, quantity=command))
+        print(f"{item_id} -- {command}")
+        return cart_id, None, ""
+
     # Handle adding a new item
     elif cart_id is not None:
         # Check if the upc is in the database
-        if session.scalar(select(Item).filter(Item.upc == int(command))).one_or_none():
-            return cart_id, item_id, ""
+        if command == "e":
+            error = checkout(session=session, cart_id=cart_id)
+            return None, None, error
         else:
-            return cart_id, "", "ERROR: Code not found"
+            if session.scalar(select(Item).filter(Item.upc == int(command))):
+                item_id = command
+                print(f"selecting {item_id}")
+                return cart_id, item_id, ""
+            else:
+                return cart_id, "", "ERROR: Code not found"
 
     else:
         cart_id, item_id if item_id else "", ""
@@ -181,7 +209,7 @@ def main(db_uri: str, location: int):
                 else:
                     prompt = f"[{item_id}# quantity]>"
             elif cart_id and not item_id:
-                prompt = f"[{cart_id}# code]> "
+                prompt = f"[{cart_id}# code, e for chekout]> "
             else:
                 prompt = "#> "
 
@@ -210,6 +238,7 @@ def main(db_uri: str, location: int):
                         db_remote_uri=db_uri,
                         session=session,
                     )
+                    print(cart_id, item_id, error)
 
     print("GoodBye!")
 

src/retailtwin/cli/pos/models.py

@@ -33,7 +33,10 @@ class Item(Base):
     __tablename__ = "items"
     upc: Mapped[int] = mapped_column(primary_key=True)
     unitprice: Mapped[Decimal] = mapped_column(Numeric(9, 2), nullable=False)
-    discount: Mapped[Dict[str, Dict[str, int]]] = mapped_column(JSON, nullable=True)
+    discount_name: Mapped[str]
+    discount_definition: Mapped[Dict[str, Dict[str, int]]] = mapped_column(
+        JSON, nullable=True
+    )
 
 
 class Customer(Base):

src/retailtwin/cli/stock.py

@@ -64,14 +64,14 @@ def handle_command(command: str, **kwargs):
                 pl.col("upc"),
                 pl.col("name"),
                 pl.col("package"),
-                pl.col("price"),
+                pl.col("unitprice"),
                 pl.col("best_until"),
                 pl.col("quantity"),
             ]
         ).filter(pl.col("upc") == upc)
         console.print(
             df_to_table(
-                df.with_columns(pl.col("price").cast(str)),
+                df.with_columns(pl.col("unitprice").cast(str)),
                 title=f"Item {upc} on location {location}",
             )
         )

src/retailtwin/sql/add_item_to_cart.sql

@@ -1,9 +1,9 @@
-create or replace procedure add_item_to_cart(cart integer, sku integer, quantity integer)
+create or replace procedure add_item_to_cart(cart integer, upc integer, quantity integer)
 language sql
 as
 $$
 INSERT INTO itemsoncart
-("cart", "sku", "quantity")
+("cart", "upc", "quantity")
 values
 ($1, $2, $3);
 $$;

src/retailtwin/sql/sync.py

@@ -3,7 +3,7 @@ from sqlalchemy import create_engine, text
 from sqlalchemy.orm import Session
 import retailtwin
 
-PACKAGE_ROOT = Path(retailtwin.__file__).parent / "retail"
+PACKAGE_ROOT = Path(retailtwin.__file__).parent
 
 
 def funcandproc(db_uri: str):
@@ -29,7 +29,7 @@ def funcandproc(db_uri: str):
             view_name = predicate.stem.removesuffix(".sql")
             with predicate.open() as sql:
                 # First remove the view
-                session.execute(text(f"drop view {view_name}"))
+                session.execute(text(f"drop view if exists {view_name}"))
                 session.commit()
 
                 # And sync it

src/retailtwin/sql/views/inventory.sql

@@ -9,6 +9,7 @@ create or replace view inventory as (
 			date(b.best_until at time zone 'UTC') as best_until,
 			i.quantity as quantity,
 			i."location" as "location",
+			d.name as discount_name,
 			d.definition::text as discount_definition
 		from itemsonshelf i
 			join batches b on i.batch = b.id