Engineering Circle Meeting 2024-09-02

<h1>Engineering Circle Meeting 2024-09-02 </h1>

<h2>Table of Contents</h2>

- [Attendees](#attendees)
- [Todo](#todo)
- [Main Content](#main-content)

---

## Attendees

- Kristof
- Ahmed
- Scott
- Mik
- Lee
- Jan

## Todo

- Farmerbot disclaimer
  - situation
    - farmers need to acknolwedge that Farmerbot issues aren't the responsibility of TF
    - Scott is overseeing this
- Major updates
  - need a stricter process to avoid problems
    - Whitelist
    - Jan and Maxime will check how to build the new ZOS properly
      - make sure it works on newer Linux 24.04
- ZOS hardware and software
  - current zos running on old hardware
  - need to test on different hardware
    - Sabrina is checking this with Belgium team 
- grid stack
  - load balancing
    - WIP by Bert
  - validator code
    - WIP by Bert
- VM issues reported: can ping but not SSH
  - Scott will make a ticket to see what is the issue+fix
- Dagus demo mention in forum 3.15 update post
  - remove it from post (Mik overseeing this)

## Main Content

The following are draft notes. See Todo section for specific tasks.

- grid full stack
  - done
    - grid backend stack
    - hub & bootstrap hub
  - WIP
    - load balancing
    - validator code
- TFChain
  - being deployed today
- Updates
  - have a specific process to do updates
  - to avoid:
    - nothing automatically can go in production ever
    - always work with roll-out mode
  - details
    - over next days, update process
        - set up a blocker created now an roll-out
          - start on 10 machines
        - specs:
          - no automatic updates
          - full control over the updates
        - e.g. 
          - zos need to check something, check on GH, etc.
            - goal: to have more control defined
      - proposition
        - something simple
          - e.g. whitelist
            - if you're not on the whitelist, you can't upgrade
            - need multiple signings
            - then put version
- zos
  - need simpler zos
  - status
    - tried to deploy on hetzner machine, new hardware
      - zos couldn't use the new hardware
    - suggestion
      - new base image for zos
        - used for V3-V4
          - some binaries got conflicted
      - need to run on more updated hardware/software
        - do with 24.04, instead of 20.04, or else
        - not sure with beyond 20.04 because of g-lipsi(?)
      - specs
        - we need a build system 
          - to know we can upgrade kernel and ubuntu and get an image
- current zos running on old hardware
- need to test on different hardware
- 1. test zos
- 2. rebuild zos
- farmerbot
  - statement:
    - use own responsibility
  - TODO: Scott will check
- Issues reported
  - VMs stopped to accept SSH connection
    - pingable but can't get into SSH connection
      - hypothesis
        - operator/user error
          - but then got another case
    - situation
      - user using the VM inside it
        - stopped SSH during the session
- Sentry integration
  - done in typescript
  - being worked on for Go
    - in standby