Lately I have been using AI more and more in my codebase and that’s been a bit of a hit and miss if you ask me.

I reckon it’s an amazing tool that allows developers to truly optimise their workflow, however at times laziness take over and code reviews are not as frequent as they should be.

I really REALLY wanted to build something without using AI after having spent months and months getting a bit too complacent. And this is the first idea that came to mind. Of course to celebrate this occasion I went for a technology I hadn’t used in almost a decade: jQuery!!!

You can find the repo here: https://github.com/fccview/ackchyually-ai
You can play around with the tool on fccview.github.io/ackchyually-ai

Few disclaimers:

  • I am a full stack javascript developer (with extensive knowledge of php and a very good grasp of shell). This means majority of the detection will be revolving around javascript, if you develop in a different language and you feel like you know AI patterns in that language PLEASE do create a pull request so we can make the detection even more accurate for other type of syntax!
  • The tool uses MY OWN metrics, this is doing what I usually do to see if something is (in my opinion) been made with AI and barely reviewed by a human. It’s not gospel, it’s not a scientific method, it’s most definitely going to be flawed, so keep it with a grain of salt, it can always be improved and collaboration/community support will definitely help with it
  • I don’t hate AI, on contrary I enjoy using it and I find value in not needing to “google it” whenever I hit a blocker, more times than not, AI has a valid solution that can be used, refined and applied properly. What I have mostly an issue with is people randomly generating code, not reviewing it and trying to sell you the moon when they barely understand the architecture of what they built
  • Please do not make this post a fight between pro/against AI individuals, that’s not the aim of it

All the patterns I am using to identify possible AI code are here: https://github.com/fccview/ackchyually-ai/tree/main/data if you come up with more patterns/ideas and have more ideas on what to look for please reply here, open an issue and/or make a pull request, but at ONE CONDITION, do NOT use AI for this. It’d go against the core principle of this one pure little app I decided to build <3

  • irmadlad@lemmy.world
    link
    fedilink
    English
    arrow-up
    2
    ·
    3 months ago

    I share some of the same views you do about AI. I do use AI to help me with code. Not to develop and sell, just for my own personal use. So I tested out your ackchyually-ai. I had Grok create a docker compose to combine BorgBackup and BorgUI.

    Result:

    spoiler
    #########################
    # Services
    #########################
    services:
    
      # -------------------------------------------------
      #  Borg repository (data only)
      # -------------------------------------------------
      borg:
        image: ghcr.io/borgbackup/borg:latest
        container_name: borg_repo
        restart: always
        command: ["sleep", "infinity"]               # keep container alive
        volumes:
          - ./repo:/var/borg/repo                    # persistent repo
          - ./ssh:/root/.ssh:ro                     # optional SSH keys (read‑only)
        environment:
          # Uncomment if you want the repo to be encrypted by default
          # BORG_PASSPHRASE: "${BORG_PASSPHRASE}"
        networks:
          - backup_net
    
      # -------------------------------------------------
      #  Borgmatic – scheduled backup runner
      # -------------------------------------------------
      borgmatic:
        image: ghcr.io/borgmatic-collective/borgmatic:latest
        container_name: borgmatic
        restart: always
        depends_on:
          - borg
        volumes:
          - ./repo:/var/borg/repo                    # same repo as above
          - ./ssh:/root/.ssh:ro                     # SSH keys (read‑only)
          - ./borgmatic/config.yaml:/etc/borgmatic/config.yaml:ro
          - ./borgmatic/source_dirs:/source_dirs:ro # directories to back up
        environment:
          - BORG_PASSPHRASE=${BORG_PASSPHRASE}
        # Run borgmatic every day at 02:30 am using the built‑in cron of the image
        command: ["cron", "-f"]
        # If you prefer a host‑side cron, replace the command with “tail -f /dev/null”
        # and schedule `docker exec borgmatic borgmatic run` from the host.
        networks:
          - backup_net
    
      # -------------------------------------------------
      #  BorgWeb – Flask UI for browsing the repo
      # -------------------------------------------------
      borgweb:
        image: ghcr.io/borgbackup/borgweb:latest
        container_name: borgweb
        restart: always
        depends_on:
          - borg
        environment:
          - BORGWEB_REPO=/var/borg/repo
          - BORG_PASSPHRASE=${BORG_PASSPHRASE}
          # Optional basic‑auth (handled by the tiny built‑in Flask auth)
          - BORGWEB_BASIC_AUTH_USER=${BORGWEB_BASIC_AUTH_USER}
          - BORGWEB_BASIC_AUTH_PASS=${BORGWEB_BASIC_AUTH_PASS}
        volumes:
          - ./repo:/var/borg/repo
        # Expose the UI directly; you can map it to any host port you like.
        # 8080 = plain HTTP (for testing only)
        # 8443 = HTTPS if you put an external TLS terminator in front.
        ports:
          - "48443:8080"
        networks:
          - backup_net
    
      # -------------------------------------------------
      #  OPTIONAL: Minimal NGINX reverse‑proxy for TLS & auth
      #  (If you already have a front‑end proxy you can skip this block)
      # -------------------------------------------------
      nginx:
        image: nginx:alpine
        container_name: nginx_tls
        restart: always
        depends_on:
          - borgweb
        volumes:
          - ./nginx/nginx.conf:/etc/nginx/conf.d/default.conf:ro
          # If you have your own cert/key files, mount them here:
          # - ./certs/fullchain.pem:/etc/ssl/certs/fullchain.pem:ro
          # - ./certs/privkey.pem:/etc/ssl/private/privkey.pem:ro
        ports:
          - "443:443"
        networks:
          - backup_net
    

    Then I ran it through your checker.

    spoiler

    Results:

    Result breakdown

    Overall score

    This is entirely made up by ME, it’s not an official metric, so please don’t take it as gospel.

    There’s a lot of AI generated code here, it’s getting dangerously close to be sloppy. The ratio of comment to code is over 20%, this automatically pushes the score to 100% Comment ratio The amount of comments in AI generated code is borderline ridiculous. Comments are good, too many (often redundant) comments are just bloat.

    34 comments for 89 lines of code. That’s a ratio of 38%. The site assumes something is 100% AI generated if the ratio goes above 20%.

    Interesting that comment code was the trigger apparently. Personally, I comment a lot mainly because my brain is shit, so it’s very helpful. I haven’t run the docker compose, but I might just do that to see if AI came close. I’m sure some of the experts here would find the compose file to be wrong in some nature or another.

    • fccview@lemmy.worldOP
      link
      fedilink
      English
      arrow-up
      2
      ·
      3 months ago

      Hey! Thank you for testing it out, I think in my head, even the most verbose of dev wouldn’t leave >20% of comments in their codebase. The percentage works on a ratio of (commentsCount / linesOfCode) * 100 so it doesn’t just flag “a lot of comments”, it mostly checks for “too many comments”, that said, the “use common sense” at the top needs to be taken quite seriously, for example if there’s a majority of comments but none of the comments feel like written by AI, it’s clearly just the developer being verbose :)

      p.s. I find AI is pretty damn good at making docker compose files, it’s probably gonna work just fine <3

      • irmadlad@lemmy.world
        link
        fedilink
        English
        arrow-up
        3
        ·
        3 months ago

        I think in my head, even the most verbose of dev wouldn’t leave >20% of comments in their codebase.

        Well, I guess that’s where it’s pretty clear that I’m no dev. LOL Give me a # key and I’ll go nuts. At my age, and with other issues, I need all the help I can get. It’s a pretty cool tool ackchyually. I tested out a few more compose files last night without commenting and they seemed to pass the test. Thanks for sharing. It was a pretty fun experience.

        • fccview@lemmy.worldOP
          link
          fedilink
          English
          arrow-up
          2
          ·
          3 months ago

          Aw thank you for the kindness ♥️ There’s nothing wrong in verbosity btw, I may have been a bit too harsh on my parameters, I’ll tweak it a little :)