Automate Your Work

Making a computer do a job that a person used to do, step by step, is called automation. A program that does a small job of this kind, often written in an afternoon and run from the command line, is called a script. You have written programs since Grade 8; a script is just a program written to save you work.

Some jobs are made for a script, and some are not. A job suits a program when:

• there is a rule. “Name each photo by the date in its name” is a rule. “Choose the best photo” is not: it needs judgment.
• there are many of them. Six files are quicker by hand. Six hundred are not.
• it comes back. A job done every week or every month repays the time spent on the program again and again.
• mistakes are likely by hand. People typing the 200th name make mistakes that a program doesn’t.

Then do the sum. Say joining the section teachers’ fee lists takes you 2 hours by hand every month. A program to do it takes 5 hours to write and test, and then 10 minutes a month to run and check its results.

• Each month, the program saves 2 hours − 10 minutes = 1 hour 50 minutes.
• The 5 hours are paid back after 5 ÷ 1.83 ≈ 2.7 months: from the third month, you are ahead.
• Over a 10-month school year: by hand, 20 hours; with the program, 5 + 10 × 10 minutes ≈ 6 hours 40 minutes.

The time isn’t the only thing to count. A program never gets tired, and it does the job the same way every time. But it can also make a mistake the same way every time. Count the time to check its results, and to fix it when the job changes.

In Grade 6 you saw that files sit in folders, and folders in folders, in a tree. In Grade 10 you met paths, the way to a file through that tree, such as school/class10/marks.csv, and the working directory, the folder a program runs in. Python’s standard library module pathlib turns a path into an object (Unit 1) with attributes and methods for everything you do with files.

A folder to practice on

The Python on this page runs inside your browser, so it can’t see the files on your phone or computer. That is a good thing: a program on a web page that could rename your photos would be dangerous. Instead, it has a small folder of its own, which disappears when you leave the page. This module, samples.py, makes a folder of empty files with the names a phone’s camera gives them, so the programs in this section have something to work on. On a computer, you would point the same programs at a real folder (a copy of one, as you will see).

"""Sample files for this unit's programs: a phone's Camera folder, made fresh."""from pathlib import Path CAMERA = ["IMG_20261014_093012.jpg", "IMG_20261014_093047.jpg", "IMG_20261014_101500.jpg",          "PXL_20261014_101500123.jpg", "IMG-20261015-WA0003.jpg", "IMG-20261015-WA0004.jpg",          "VID_20261020_180501.mp4", "IMG_20261102_071533.jpg",          "Screenshot_20261103-214410.png", "notes.txt"]  def make_camera(folder="Camera"):    """Make a folder of empty files with a phone's names, and return its path."""    path = Path(folder)    path.mkdir(exist_ok=True)    for name in CAMERA:        (path / name).write_text("")      # an empty file: only its name matters here    return path  def tree(folder, indent=""):    """Print what is in a folder, and in the folders inside it."""    for item in sorted(Path(folder).iterdir()):        print(indent + item.name + ("/" if item.is_dir() else ""))        if item.is_dir():            tree(item, indent + "    ")

tree() is recursive, like Unit 2’s folder sizes: a folder’s tree is its own items, then the tree of each folder inside it.

This program makes the folder, then looks at every file in it and counts the files of each type:

import samples camera = samples.make_camera()kinds = {}for item in sorted(camera.iterdir()):    print(item.name, "| stem:", item.stem, "| suffix:", item.suffix)    kinds[item.suffix] = kinds.get(item.suffix, 0) + 1print(kinds)

IMG-20261015-WA0003.jpg | stem: IMG-20261015-WA0003 | suffix: .jpgIMG-20261015-WA0004.jpg | stem: IMG-20261015-WA0004 | suffix: .jpgIMG_20261014_093012.jpg | stem: IMG_20261014_093012 | suffix: .jpgIMG_20261014_093047.jpg | stem: IMG_20261014_093047 | suffix: .jpgIMG_20261014_101500.jpg | stem: IMG_20261014_101500 | suffix: .jpgIMG_20261102_071533.jpg | stem: IMG_20261102_071533 | suffix: .jpgPXL_20261014_101500123.jpg | stem: PXL_20261014_101500123 | suffix: .jpgScreenshot_20261103-214410.png | stem: Screenshot_20261103-214410 | suffix: .pngVID_20261020_180501.mp4 | stem: VID_20261020_180501 | suffix: .mp4notes.txt | stem: notes | suffix: .txt{'.jpg': 7, '.png': 1, '.mp4': 1, '.txt': 1}

Two things to notice. sorted() puts the paths in the order of their characters’ codes (Grade 7), so IMG- comes before IMG_ (a hyphen’s code is smaller than an underscore’s), and every capital letter before every small one, which is why notes.txt comes last. And the name tells you a lot: IMG_20261014_093012 was taken on 14 October 2026 at 09:30:12. A phone camera names its photos by the moment they were taken. That is the rule a program can use.

Renaming by a rule

Names like IMG_20261014_093012.jpg sort by date, as Grade 8 showed names should, but they are hard to read. A better name says the date and time plainly: 2026-10-14 09.30.12.jpg. Year first, then month, then day, so the names still sort in time order. (Dots, not colons, between the hours, minutes and seconds: Windows doesn’t allow : in a file’s name.)

The rule, for names made by this camera: the date is the 8 characters after IMG_, the time the 6 after the next underscore. String slices (Grade 9) cut them out:

IMG_20261014_093012.jpg    ^^^^^^^^ ^^^^^^    [4:12]   [13:19]     the type is [19:]

A script that changes things should first show what it would do, without doing it: a dry run. Here the constant DRY_RUN decides. Run it as it is, then read the plan:

import samples DRY_RUN = True      # True: only show what would happen. False: really do it.  def new_name(name):    """IMG_20261014_093012.jpg → 2026-10-14 09.30.12.jpg, or None for other names."""    if not (name.startswith("IMG_") and len(name) == 23):        return None    date, time, kind = name[4:12], name[13:19], name[19:]    return f"{date[:4]}-{date[4:6]}-{date[6:]} {time[:2]}.{time[2:4]}.{time[4:]}{kind}"  camera = samples.make_camera()for item in sorted(camera.iterdir()):    new = new_name(item.name)    if new is None:        print("skip:", item.name)        continue    month = camera / new[:7]                  # a folder for the month: Camera/2026-10    target = month / new    if target.exists():        print("skip:", item.name, "(" + new + " is taken)")        continue    print(item.name, "->", target)    if not DRY_RUN:        month.mkdir(exist_ok=True)        item.rename(target) if not DRY_RUN:    samples.tree(camera)

skip: IMG-20261015-WA0003.jpgskip: IMG-20261015-WA0004.jpgIMG_20261014_093012.jpg -> Camera/2026-10/2026-10-14 09.30.12.jpgIMG_20261014_093047.jpg -> Camera/2026-10/2026-10-14 09.30.47.jpgIMG_20261014_101500.jpg -> Camera/2026-10/2026-10-14 10.15.00.jpgIMG_20261102_071533.jpg -> Camera/2026-11/2026-11-02 07.15.33.jpgskip: PXL_20261014_101500123.jpgskip: Screenshot_20261103-214410.pngskip: VID_20261020_180501.mp4skip: notes.txt

Nothing has moved: it is a plan. Read it before you trust it. Four photos would move, each into a folder for its month; six files would be skipped, because the rule knows only one kind of name. That is safe (a skipped file is still there, untouched), but the plan shows the rule is too narrow. You will widen it with a regular expression later in this unit.

Never overwrite

Look at line 22: the program checks whether the new name is already taken before it renames. Why bother? Run this and see what rename does when the new name is taken:

from pathlib import Path Path("plan.txt").write_text("The real plan for the exhibition")Path("draft.txt").write_text("An old draft")Path("draft.txt").rename("plan.txt") print(Path("plan.txt").read_text())print(Path("draft.txt").exists())

An old draftFalse

No error, no warning: the real plan is gone, replaced by the old draft. On Linux and macOS (and in this page’s Python), renaming onto a name that is taken silently replaces the file that had it. On Windows, the same line stops with a FileExistsError instead. A script must not depend on which computer it runs on: check first, and skip (or choose another name) if the name is taken.

Lists of students, fees, marks and stock usually live in spreadsheets. In Grade 7 you cleaned a spreadsheet by hand; in Grade 9 you read a table from a CSV file in Python, and in Grade 10 you cleaned one with pandas. Here the job is the one an office really has: several sheets, from several people, each typed in its own way, joined into one clean list, every month.

A program reads a spreadsheet most easily as a CSV file (comma-separated values: one row per line, the values separated by commas, Grade 9). Every spreadsheet program can save a sheet as CSV and open one. Python’s csv module reads and writes them, and handles the hard parts, like a value with a comma inside it, written in quotes: "1,500".

Three sheets, three ways

Three section teachers collected the fee for the Grade 9 science trip, Rs 1,500 each, and each kept a sheet. Here they are, saved as CSV:

Name,Phone,PaidAarati Shrestha,9841000011,1500Bibek Tamang,9811000022,1500Dawa Sherpa,9761000033,750

Student name,Parent mobile,Amount paid Gita Magar ,+977-9801000044,"1,500"Hari Thapa,98-51000055,Rs 1500Kabita Rai,984100006,1000

name,mobile,paidNabin K.C.,9861000077,१५००Pema Lama,9741000088,0Rojina Karki,984-1000099,1500

Each is fine for a person to read. To a program, they are a mess:

• The columns have different names: Name, Student name, name; Phone, Parent mobile, mobile.
• Amounts are written four ways: 1500, "1,500" (with a comma, so the CSV puts it in quotes), Rs 1500, and १५०० in Nepali digits.
• Phone numbers are written four ways, with +977 (Nepal’s country code), dashes in different places, and one (Kabita’s) with only nine digits.
• A name has spaces round it.

One rule per problem

Clean data the way Grade 10 did: look first, then one rule for each problem, each in a function you can test by itself.

import csv # each teacher's name for a column, and the name we useCOLUMNS = {"name": "name", "student name": "name",           "phone": "phone", "parent mobile": "phone", "mobile": "phone",           "paid": "paid", "amount paid": "paid"}  def clean_amount(text):    """'1500', '1,500', 'Rs 1500' or '१५००' → 1500."""    return int(text.replace("Rs", "").replace(",", "").strip())  def clean_phone(text):    """A mobile number as its ten digits, or None if it can't be one."""    digits = "".join(c for c in text if c.isdigit())    if len(digits) == 13 and digits.startswith("977"):        digits = digits[3:]                       # take off Nepal's country code    if len(digits) == 10 and digits.startswith("9"):        return digits    return None  rows, problems = [], []for section in ["9A", "9B", "9C"]:    with open(section + ".csv", encoding="utf-8-sig", newline="") as file:        for row in csv.DictReader(file):            row = {COLUMNS[key.strip().lower()]: value for key, value in row.items()}            name = row["name"].strip()            phone = clean_phone(row["phone"])            if phone is None:                problems.append(section + ", " + name + ": phone " + row["phone"])            rows.append({"section": section, "name": name,                         "phone": phone or "", "paid": clean_amount(row["paid"])}) with open("trip_fees.csv", "w", encoding="utf-8", newline="") as file:    writer = csv.DictWriter(file, fieldnames=["section", "name", "phone", "paid"])    writer.writeheader()    writer.writerows(rows) print(len(rows), "students, Rs", sum(row["paid"] for row in rows), "collected")print("Check by hand:", problems)

9 students, Rs 10750 collectedCheck by hand: ['9B, Kabita Rai: phone 984100006']

Read how each problem was handled:

• Line 28 turns every teacher’s column names into ours with the dictionary COLUMNS. It is a dictionary comprehension, like Grade 10’s list comprehension but in curly brackets, with a key: value pair for each item: it makes a new row with the same values under the new keys. A new teacher’s sheet with yet another column name stops the program with a KeyError, which is what you want: better to stop than to guess.
• clean_amount takes out “Rs”, commas and spaces. Then int() does something surprising: it reads Nepali digits too. int("१५००") is 1500, because Python counts any Unicode decimal digit as a digit.
• clean_phone keeps only the digits, c.isdigit(), takes off a 977 in front, and accepts only ten digits starting with 9. Kabita’s number has nine. The program doesn’t guess the missing digit: it writes the row with an empty phone and adds it to a list for a person to check. A cleaning program should fix only what it can be sure of, and report the rest.
• Line 26 opens the files with encoding="utf-8-sig". The next part says why.

The program also wrote trip_fees.csv, shown under its output when you run it: one clean sheet, every amount a number, every phone ten digits or empty. Open it in any spreadsheet program, or use it in the next section.

What spreadsheets do to data

Spreadsheet programs try to be helpful, and some of their help quietly changes your data. These are the traps that catch offices every day:

• The invisible first character. Excel’s “CSV UTF-8” puts three special bytes at the very start of the file, a byte order mark (BOM), which says “this file is UTF-8”. Read with plain encoding="utf-8", they become a character, , at the start of the first column’s name: the column is called "name", not "name", and row["name"] is a KeyError, though the name looks right when printed. encoding="utf-8-sig" reads a BOM if there is one and drops it, and reads a file without one just the same. Use it for every CSV a spreadsheet saved.
• Numbers that aren’t numbers. A phone number, a bank account number or a roll number is a label, not an amount. A spreadsheet that takes it for a number drops leading zeros (0012 becomes 12), and keeps only 15 significant digits: a 16-digit account number has its last digit turned into a 0 for ever. Format such columns as Text before typing or pasting them in.
• Things that look like dates. Type 1-2 or MARCH1 into many spreadsheet programs and it becomes a date.

Now write to the parents of the three students who still owe. Each letter is the same text with a few things changed: the name, the section, the amount. Joining a template (the text with gaps) with a list (one row per letter) to make one letter per row is called a mail merge. Word processors can do it, and so can six lines of Python.

Here is trip_fees.csv, as the program above wrote it:

section,name,phone,paid9A,Aarati Shrestha,9841000011,15009A,Bibek Tamang,9811000022,15009A,Dawa Sherpa,9761000033,7509B,Gita Magar,9801000044,15009B,Hari Thapa,9851000055,15009B,Kabita Rai,,10009C,Nabin K.C.,9861000077,15009C,Pema Lama,9741000088,09C,Rojina Karki,9841000099,1500

A string’s format() method fills each {name} gap with the value given for it:

import csv FEE = 1500LETTER = """Dear parent of {name} ({section}),The science trip's fee is Rs {fee}. We have received Rs {paid},so Rs {due} is still to pay. Please send it by Friday.Science Club""" with open("trip_fees.csv", encoding="utf-8-sig", newline="") as file:    for row in csv.DictReader(file):        due = FEE - int(row["paid"])        if due > 0:            print(LETTER.format(fee=FEE, due=due, **row))

Dear parent of Dawa Sherpa (9A),The science trip's fee is Rs 1500. We have received Rs 750,so Rs 750 is still to pay. Please send it by Friday.Science Club Dear parent of Kabita Rai (9B),The science trip's fee is Rs 1500. We have received Rs 1000,so Rs 500 is still to pay. Please send it by Friday.Science Club Dear parent of Pema Lama (9C),The science trip's fee is Rs 1500. We have received Rs 0,so Rs 1500 is still to pay. Please send it by Friday.Science Club

**row passes each of the row’s keys and values as a named argument, section="9A", name="Dawa Sherpa", …, so every gap named after a column is filled from it. A gap with no value given (a column that isn’t there) stops the program with a KeyError, naming the gap: better than a letter that says “Dear parent of {name}”.

Certificates, one file each

Certificates are printed, so make each one a web page: a browser shows it, and on a computer prints it (or saves it as a PDF) with the page’s own layout. Here are the exhibition’s winners:

name,grade,event,placeAarati Shrestha,9,Science Exhibition,FirstDawa Sherpa,10,Science Exhibition,Secondरोजिना कार्की,8,Science Quiz,FirstBibek & Kiran Tamang,7,Science Quiz,Second

An HTML page is full of { and } (its CSS uses them), which format() would take for gaps. So use the standard library’s string.Template, whose gaps are written $name instead. Its substitute() method fills them, and like format() stops with a KeyError if a value is missing.

import csvimport htmlfrom string import Template PAGE = Template("""<!DOCTYPE html><html><head><meta charset="utf-8"><title>Certificate: $name</title><style>body { font-family: serif; text-align: center; border: 12px double #7a5c00; padding: 40px; }</style></head><body><h1>Certificate of Merit</h1><p>This is to certify that</p><h2>$name</h2><p>of Grade $grade won <b>$place place</b> in the $event.</p></body></html>""") with open("winners.csv", encoding="utf-8-sig", newline="") as file:    rows = list(csv.DictReader(file)) made = 0for number, row in enumerate(rows, start=1):    safe = {key: html.escape(value.strip()) for key, value in row.items()}    file_name = f"certificate-{number:02}.html"    with open(file_name, "w", encoding="utf-8") as page:        page.write(PAGE.substitute(safe))    made += 1    print(file_name, "-", row["name"]) print(len(rows), "rows in,", made, "certificates out")

certificate-01.html - Aarati Shresthacertificate-02.html - Dawa Sherpacertificate-03.html - रोजिना कार्कीcertificate-04.html - Bibek & Kiran Tamang4 rows in, 4 certificates out

Run it, and the four pages appear under the output. Look at certificate-04: the & in the name has become &, which a browser shows as &. That is html.escape(): in HTML, &, < and > have special meanings, and a name with one of them could break the page, and a name written on purpose with < in it could even put a program into the page, an attack you will meet in cybersecurity. Any text you put into a web page from a list must be escaped first.

f"certificate-{number:02}.html" writes the number with two digits, 01, 02, … , so the files sort in order up to 99. (Why not name each file after the student? Because a name can contain characters a file name can’t, / for one, and two students can have the same name.)

A shop owner in Patan gets orders by chat all day. At the end of the month she copies the chat into a text file and wants every amount of money in it, and every phone number. They don’t sit in columns: they are somewhere in each message, written however the customer felt like writing them. "Rs" in line tells you a line mentions rupees, but not which characters are the amount.

What she needs is a way to describe what the text she wants looks like: “Rs, then maybe a dot, maybe a space, then digits and commas”. A description like that, written in a small language of its own, is a regular expression (often called a regex). Python’s standard library module re finds the text that matches one.

The pieces

A regular expression is made of characters that match themselves, and a few that have special meanings:

Put together, Rs\.? ?[\d,]+ means: R, s, perhaps a dot, perhaps a space, then one or more characters that are each a digit or a comma. It matches Rs 1,500, Rs.500 and Rs 75.

In Python, write a pattern as a raw string, with an r before the quotes: r"\d+". In an ordinary string, Python itself reads some backslash pairs ("\b" is a backspace character, Grade 7’s control codes), so the pattern the re module gets is not the one you wrote. In a raw string, a backslash is just a backslash.

Trying a pattern

Three functions answer the three questions you ask most:

A pattern is tested the way a program is: on examples you know the answer to, and on examples that must not match. A pattern that matches everything it should is only half tested; most bugs in regular expressions are things they match that they shouldn’t.

Finding every match

Here is the shop’s chat for one evening:

Sita: 2 kg sugar and 1 packet tea, Rs 340 sent by QR. Call 9841000011Hari: रु 1,200 बाँकी छ, will pay FridayGita: paid NPR 12,000 for the gas cylinder. My number 9801000044Ram: send 5 eggs to room 12Maya: Rs.75.50 change was missing!

Before writing the program, get the pattern right on the tester:

import re AMOUNT = re.compile(r"(?:Rs|रु|NPR)\.? ?[\d,]+(?:\.\d\d)?")MOBILE = re.compile(r"\b9[678]\d{8}\b") with open("orders.txt", encoding="utf-8") as file:    for line in file:        who = line.split(":")[0]        print(who, AMOUNT.findall(line), MOBILE.findall(line))

Sita ['Rs 340'] ['9841000011']Hari ['रु 1,200'] []Gita ['NPR 12,000'] ['9801000044']Ram [] []Maya ['Rs.75.50'] []

Three new things:

• re.compile() turns a pattern into an object that can be used again and again, with the same functions as methods: AMOUNT.findall(line). Giving it a name in capitals, at the top, also says what it is for.
• (?:…) is a group that only groups. If a pattern has ordinary ( ) groups, findall() gives what the groups matched instead of the whole match; (?: … ) groups without remembering, so findall gives the whole amount.
• \b at both ends of the mobile pattern: without them, the pattern would also find ten digits inside a longer number, such as a 16-digit bank account number.

Groups: taking a match apart

A group remembers what it matched, so one pattern can both find something and take it apart. Here is the photo-renaming rule again, as a regular expression this time:

import re PHOTO = re.compile(r"(IMG|VID|PXL)_(\d{4})(\d\d)(\d\d)_(\d\d)(\d\d)(\d\d)\d*\.(\w+)") for name in ["IMG_20261014_093012.jpg", "PXL_20261014_101500123.jpg", "notes.txt"]:    m = PHOTO.fullmatch(name)    if m:        print(m.groups())        kind, year, month, day, hour, minute, second, ext = m.groups()        print(f"  -> {year}-{month}-{day} {hour}.{minute}.{second}.{ext}")    else:        print(name, "doesn't match")

('IMG', '2026', '10', '14', '09', '30', '12', 'jpg')  -> 2026-10-14 09.30.12.jpg('PXL', '2026', '10', '14', '10', '15', '00', 'jpg')  -> 2026-10-14 10.15.00.jpgnotes.txt doesn't match

m.groups() gives what each group matched, in order, numbered from 1 by its opening bracket: m.group(2) is the year. The \d* after the seconds lets through the extra digits another phone adds (PXL_…101500123 has thousandths of a second). One pattern now does the work of the slices before, for three kinds of name instead of one.

You can also name a group, (?P<year>\d{4}), and ask for it by name: m.group("year"). In a long pattern that is much easier to read than counting brackets.

Replacing

re.sub(pattern, new, text) replaces every match with something else. It cleans up text in one line:

import re print(re.sub(r"\D", "", "+977-980-1000044"))           # \D: anything that isn't a digitprint(re.sub(r"\s+", " ", "  Gita    Magar  ").strip())  # any run of spaces → one spaceprint(re.sub(r"(\d{4})/(\d\d?)/(\d\d?)", r"\1-\2-\3", "Paid on 2083/7/15"))

9779801000044Gita MagarPaid on 2083-7-15

The last line uses the groups in the replacement: \1 is what group 1 matched. It rewrites dates written with slashes as dates written with dashes. (The ? after the second \d lets a month or day have one digit or two.)

Regular expressions and Nepali

Python’s re module works on Unicode text (Grade 7), so Nepali works, with two surprises:

import re print(re.findall(r"\d+", "फोन ९८४१०००० र 9851000022"))print(int("९८४१"))print(re.findall(r"\w+", "नमस्ते साथी"))print(re.findall(r"[ऀ-ॿ]+", "नमस्ते साथी"))

['९८४१००००', '9851000022']9841['नमस', 'त', 'स', 'थ']['नमस्ते', 'साथी']

• \d matches Nepali digits too, and int() reads them. Usually that is what you want. If a pattern must accept only 0 to 9 (an account number another program will read), write [0-9] instead of \d.
• \w+ breaks Nepali words. To Python, \w is a letter, a digit or _, and Devanagari’s vowel signs (ा, ी, े) and the halanta (्) count as marks, not letters. So “नमस्ते” falls apart where its marks are. For Nepali words, match the Devanagari block of Unicode, [ऀ-ॿ]+ (U+0900 to U+097F, Grade 7’s code points), or simply split on spaces.

In Grade 10 your program asked a weather API for Bhaktapur’s forecast, and in Unit 6 you saw what such a request is: HTTP, a status code, an answer in JSON, sent with urllib. Collecting data from the web is a big part of automation: today’s prices, the week’s rainfall, notices on an office’s website. There are three ways to get it, and the order matters:

1. An official API (Unit 6): the site’s own way for programs to ask for data. It is documented, it answers in JSON, and the site wants you to use it. Always look for one first.
2. A file to download: many organizations publish their data as CSV or spreadsheet files. One download, then it is all yours to clean, as in this unit.
3. Reading the web page itself: a program that downloads pages made for people and picks the data out of their HTML, called web scraping. Use it only when there is no API and no file, and only where the site allows it.

Good manners

A program can ask a website for pages far faster than any person, and a small website, a school’s or a ward office’s on a cheap server, can be slowed down or knocked over by one careless script. So a program that collects from the web follows rules, as a polite visitor would:

• Read the site’s terms of use. Some forbid automatic collection, or allow it only for some purposes. Follow them.
• Read its robots.txt. Most websites have a file at /robots.txt (for example https://www.example.org/robots.txt) that says which parts of the site programs may visit. Your program should check it, and stay out of what it forbids.
• Go slowly. One request at a time, with a pause between them: a few seconds, or what the site asks for. Never ask for the same page twice: save what you got (a cache) and use the saved copy.
• Say who you are. Every request carries a User-Agent header (Unit 6) naming the program that sent it. Give yours a name that says what it is and how to reach the person responsible, so that a worried site owner can contact you instead of blocking you.
• Don’t collect people. Names, phone numbers and photos of people are personal data, protected by Nepal’s Privacy Act, 2075 (2018), and by the same idea of privacy you met in Grade 8. A program that gathers them from websites is doing something its makers can be held responsible for, even when the information was public.
• Respect copyright. That data can be read doesn’t mean it can be republished. Credit the source, as Open-Meteo’s license asks.

A robots.txt file is plain text, written for programs. Here is one for a made-up school’s website:

User-agent: *Disallow: /admin/Disallow: /searchCrawl-delay: 5 User-agent: GreedyBotDisallow: /

User-agent: * starts the rules for every program; Disallow: names the start of addresses they may not visit; Crawl-delay: 5 asks for 5 seconds between requests (Crawl-delay isn’t part of the written standard, below, so some programs ignore it; a polite one doesn’t). The second group forbids one program, GreedyBot, the whole site. Python’s urllib.robotparser reads the rules and answers the question for you:

from urllib.robotparser import RobotFileParser robots = RobotFileParser()with open("robots.txt", encoding="utf-8") as file:    robots.parse(file.read().splitlines()) SITE = "https://school.example"for path in ["/notices", "/notices/holiday.pdf", "/search?q=exam", "/admin/login"]:    print(path, robots.can_fetch("NoticeBot", SITE + path), robots.can_fetch("GreedyBot", SITE + path))print("Wait between requests:", robots.crawl_delay("NoticeBot"), "seconds")

/notices True False/notices/holiday.pdf True False/search?q=exam False False/admin/login False FalseWait between requests: 5 seconds

On a computer, robots.set_url(SITE + "/robots.txt") and robots.read() fetch the real file instead of reading a saved one.

An API that answers for many places at once

Say you want three days’ forecast for five cities in a spreadsheet. A careless program asks the weather API five times. A careful one reads the documentation first, and Open-Meteo’s says that the latitude and longitude may each be a list, separated by commas: then the answer is a JSON list, one forecast for each place. Five places, one request. (Open-Meteo still counts it as five calls toward its daily limit, but your program asks and waits once instead of five times.)

This program uses Grade 10’s module internet.py, which fetches an address on this page or on a computer alike:

"""Fetching text from the internet, on this page or on a computer."""  def get_text(address):    """The text found at a web address, such as an API's JSON answer.    Raises OSError if the address can't be reached."""    try:        from pyodide.http import open_url        # Python in a web page        from pyodide.ffi import JsException    except ImportError:        from urllib.request import urlopen       # Python on a computer        with urlopen(address) as answer:            return answer.read().decode("utf-8")    try:        return open_url(address).read()    except JsException:                          # the browser couldn't fetch it:        raise OSError("Couldn't reach " + address)   # the error a computer gives

It needs the internet. Run it, and it writes forecast.csv, shown under its output:

import csvimport jsonimport internet PLACES = {"Kathmandu": (27.71, 85.32), "Pokhara": (28.21, 83.99), "Biratnagar": (26.45, 87.27),          "Nepalgunj": (28.05, 81.62), "Dhangadhi": (28.70, 80.59)} latitudes = ",".join(str(lat) for lat, lon in PLACES.values())longitudes = ",".join(str(lon) for lat, lon in PLACES.values())address = ("https://api.open-meteo.com/v1/forecast?latitude=" + latitudes           + "&longitude=" + longitudes           + "&daily=temperature_2m_max,precipitation_sum&timezone=Asia/Kathmandu&forecast_days=3") try:    answers = json.loads(internet.get_text(address))     # one request: a list, one answer per placeexcept OSError:    print("Couldn't reach the weather API. Are you online?")else:    with open("forecast.csv", "w", encoding="utf-8", newline="") as file:        writer = csv.writer(file)        writer.writerow(["place", "date", "highest_c", "rain_mm"])        for name, answer in zip(PLACES, answers):            daily = answer["daily"]            for row in zip(daily["time"], daily["temperature_2m_max"], daily["precipitation_sum"]):                writer.writerow([name, *row])    print("Wrote forecast.csv:", len(answers), "places, 3 days each")    print("Weather data by Open-Meteo.com")

Wrote forecast.csv: 5 places, 3 days eachWeather data by Open-Meteo.com

zip(PLACES, answers) pairs each place’s name with its answer, in the order they were asked; [name, *row] makes a row of the name followed by the three values in row. Open forecast.csv in a spreadsheet, and it is a table ready to chart (Grade 10). Run it again tomorrow, and you have tomorrow’s forecast: that is the kind of chore a script does every morning without being asked (the command line section shows how).

Reading a page’s HTML

When there is no API, a page’s HTML still has a structure (Grade 8): the notices are rows of a table, each title inside a link. Here is a made-up school’s notice page, saved as a file:

<!DOCTYPE html><html><head><meta charset="utf-8"><title>Notices</title></head><body><h1>Notices</h1><table class="notices">  <tr><th>Date</th><th>Notice</th></tr>  <tr><td>2083-07-05</td><td><a href="/notices/scholarship.pdf">Scholarship forms for Grade 11</a></td></tr>  <tr><td>2083-07-02</td><td><a href="/notices/holiday.pdf">Holiday for Tihar</a></td></tr>  <tr><td>2083-06-28</td><td><a href="/notices/routine.pdf">First term exam routine</a></td></tr></table></body></html>

A regular expression is the wrong tool for HTML: tags can be written in many ways and nested inside each other (the fact above). The standard library has a parser for it, html.parser. You write a class that inherits from its HTMLParser (Unit 1’s inheritance), and the parser calls your methods as it reads: handle_starttag() at each opening tag, handle_endtag() at each closing one, handle_data() for the text between.

from html.parser import HTMLParserfrom urllib.parse import urljoin  class NoticeReader(HTMLParser):    """Collects each row of the notices table: its date, title and link."""     def __init__(self):        super().__init__()        self.notices = []        self.cells = []          # the text of each cell in the row being read        self.link = None        self.in_cell = False     def handle_starttag(self, tag, attrs):        if tag == "tr":            self.cells, self.link = [], None        elif tag == "td":            self.in_cell = True            self.cells.append("")        elif tag == "a":            self.link = dict(attrs).get("href")     def handle_endtag(self, tag):        if tag == "td":            self.in_cell = False        elif tag == "tr" and len(self.cells) == 2:            self.notices.append((self.cells[0], self.cells[1], self.link))     def handle_data(self, data):        if self.in_cell:            self.cells[-1] += data.strip()  reader = NoticeReader()with open("notices.html", encoding="utf-8") as file:    reader.feed(file.read()) for date, title, link in reader.notices:    print(date, "|", title, "|", urljoin("https://school.example/notices", link))print(len(reader.notices), "notices")

2083-07-05 | Scholarship forms for Grade 11 | https://school.example/notices/scholarship.pdf2083-07-02 | Holiday for Tihar | https://school.example/notices/holiday.pdf2083-06-28 | First term exam routine | https://school.example/notices/routine.pdf3 notices

The header row has <th> cells, not <td>, so it has no cells and is left out. urljoin() turns the link in the page, /notices/scholarship.pdf, which starts from the site’s root, into a full address.

On a computer, the same program reads the live page: fetch it politely (below), then reader.feed(text).

"""Fetching pages politely: slowly, saying who we are, and never twice."""import timefrom pathlib import Pathfrom urllib.request import Request, urlopen  class PoliteFetcher:    def __init__(self, agent, wait, folder="cache"):        self.agent = agent              # who we are, for the site's owner        self.wait = wait                # seconds between requests        self.folder = Path(folder)        self.folder.mkdir(exist_ok=True)        self.last = 0.0                 # when we last asked     def get(self, address, name):        """The text at an address, from our saved copy if we have one."""        saved = self.folder / name        if saved.exists():            return saved.read_text(encoding="utf-8")        pause = self.last + self.wait - time.monotonic()        if pause > 0:            time.sleep(pause)           # never faster than one request per wait        request = Request(address, headers={"User-Agent": self.agent})        with urlopen(request, timeout=15) as answer:            text = answer.read().decode("utf-8")        self.last = time.monotonic()        saved.write_text(text, encoding="utf-8")        return text  fetcher = PoliteFetcher("NoticeBot/1.0 (a Grade 11 school project; ask the computer club)", wait=5)

Every rule of good manners is in it: the pause (time.sleep), the name (User-Agent), the cache (a page saved once is never fetched again), and a timeout, so it never waits for ever (Unit 6). Check robots.txt before calling get().

In Grade 10 you ran programs from the command line: python3 results.py. A script becomes much more useful when you can tell it what to work on, in the same command: python3 tidy_photos.py Camera, for one folder; python3 tidy_photos.py Pictures, for another; --move to really do it. The words after the program’s name are its command-line arguments.

Python puts them in a list, sys.argv (“argument values”), in the sys module: sys.argv[0] is the program’s own name, and the rest are the words that followed it. A program that prints them:

import sysprint(sys.argv)

sita@office-pc:~$ python3 show_args.py Camera --move['show_args.py', 'Camera', '--move']

For anything more than one argument, use the standard library’s argparse. You describe the arguments; it reads sys.argv, checks it, writes a help message, and gives clear errors, all by itself:

• a positional argument, parser.add_argument("folder"), must be given, in its place;
• an option, parser.add_argument("--move", action="store_true"), may be given or not: args.move is True if it was.

Here is the photo program again, finished: the regular expressions, the dry run, never overwriting, and arguments. Without --move it only shows what it would do: the safe thing is what happens when you forget.

"""Sort a phone's photos and videos into a folder for each month, named bywhen they were taken. It only shows what it would do, unless you add --move."""import argparseimport refrom pathlib import Path # IMG_20261014_093012.jpg, VID_20261020_180501.mp4, PXL_20261014_101500123.jpgCAMERA = re.compile(r"(IMG|VID|PXL)_(\d{4})(\d\d)(\d\d)_(\d\d)(\d\d)(\d\d)\d*\.(\w+)")# IMG-20261015-WA0003.jpg: a photo saved from a chat appCHAT = re.compile(r"(IMG|VID)-(\d{4})(\d\d)(\d\d)-WA(\d+)\.(\w+)")  def new_place(name):    """Where a file goes, such as "2026-10/2026-10-14 09.30.12.jpg", or None."""    m = CAMERA.fullmatch(name)    if m:        kind, y, mo, d, h, mi, s, ext = m.groups()        return f"{y}-{mo}/{y}-{mo}-{d} {h}.{mi}.{s}.{ext.lower()}"    m = CHAT.fullmatch(name)    if m:        kind, y, mo, d, n, ext = m.groups()        return f"{y}-{mo}/{y}-{mo}-{d} chat {n}.{ext.lower()}"    return None  parser = argparse.ArgumentParser(description="Sort a phone's photos into a folder for each month.")parser.add_argument("folder", help="the folder of photos")parser.add_argument("--move", action="store_true",                    help="really move the files (without it, only show what would happen)")args = parser.parse_args() folder = Path(args.folder)if not folder.is_dir():    print("There is no folder called", args.folder)    raise SystemExit(1) planned = set()moving = skipped = 0for item in sorted(folder.iterdir()):    if not item.is_file():        continue    place = new_place(item.name)    if place is None:        print("skip:", item.name, "(not a name it knows)")        skipped += 1    elif place in planned or (folder / place).exists():        print("skip:", item.name, "(" + place + " is taken)")        skipped += 1    else:        print(item.name, "->", place)        planned.add(place)        moving += 1        if args.move:            (folder / place).parent.mkdir(exist_ok=True)            item.rename(folder / place) if args.move:    print("Moved", moving, "files, skipped", skipped)else:    print("Would move", moving, "files, skip", skipped, "- add --move to do it")

Notice planned (line 37): a set of the places already given out in this run. Without it, the dry run couldn’t see that two photos taken in the same second, on two phones, would get the same name: neither file exists yet when the plan is made. With it, the dry run and the real run say exactly the same thing. (This program can’t run on the page’s Run button, because the page has no command line to give it arguments; the practice terminal below runs it.)

An AI chatbot (Grade 8, Grade 10) is good at writing exactly this kind of script, and especially regular expressions, which are hard for people to read and write. Used well, it saves real time. Used carelessly, it hands you a program that renames 2,000 files wrongly in a second. The rules you know already from Grade 10 apply, more strictly than ever, because these programs change things:

1. Say exactly what you want, with examples: the file names, a few rows of the sheet, what should and shouldn’t match.
2. Read the code before you run it, and ask about any line you don’t understand. A script you can’t explain is a script you can’t trust.
3. Test the pattern on lines that should match, and lines that shouldn’t.
4. Dry run, on a copy. Never let a script you were given touch the only copy of anything.
5. Be honest: if a program for schoolwork was written with an AI’s help, say so, as your school’s rules require.

A student asked a chatbot for “a regular expression for dates in our school notices”. It answered \d{4}-\d{2}-\d{2}. Test it before you use it:

Find a real, boring, repeated job, in the school office, a family shop, a club or your own phone, and automate it. You need a computer with Python 3 (Grade 8 showed how to install it; Grade 10 how to use the command line).

With only a phone: do steps 1, 2 and 7 on paper or in a notes app. For steps 3 to 6, use this page: press Edit on the program in this unit closest to your chore, put your own data in its place (for a CSV, edit the shown file’s rows inside the program as a string, or change the program’s list of rows), and Run. The page’s Python can’t reach your phone’s files, so the program works on its own copy, which is the safest way to test anyway. If your school has a computer lab, do the last run there.

1. Choose the chore, and ask the person who does it now to show you how. Write the steps down. Time them doing it once.
2. Is it worth it? Estimate how often it is done and how long your program will take to write and test. Do the sum from the start of this unit.
3. Get the data, with permission: a copy of the files, the sheet, or the list. Never work on the only copy. If it has people’s personal details, use it only for this, keep it safe, and delete your copy when you finish.
4. Write the program in small parts, testing each: read, clean, check, write. Report what it can’t fix instead of guessing.
5. Dry run on the copy. Read the plan with the person who does the job: is it right?
6. Run it, check the results, and count: as many out as in.
7. Hand it over: a short note (a README) saying what it does, how to run it (the exact command), what it skips and why, and how to undo it. Time it doing the job, and compare with step 1.

Ideas:

• Certificates for a school event from the list of winners or participants (the certificate program above), printed.
• A shop’s monthly report: each day’s sales sheet joined into one, a total per item and per week.
• Fee reminders: the sections’ lists joined, the amounts cleaned, a letter for each student who owes.
• Photo tidying for a school event, run on a copy of the folder.
• A weather log: your town’s forecast saved to a CSV every morning from the API, and after a month, compared with what really happened.