Artificial Intelligence

Spring 2019 -- Peter Brooks

The (ever-changing) Syllabus and readings

Decision Trees Simple Datasets
Datasets
Simple Trees
Answering the "Waiting for a table" dataset
Return your textbook (Machines That Think) immediately to me.  
Version 3.5 of your TicTacToe competitors due Fri, 6/14, midnight You should test your competitor inside this framework.
Theory of Information Here are my notes on the Theory.
A web server of one's own: Installation and Usage of the Python-Web-Server (pws).
AI and unemployment:

Essay due on Sat, June 1, midnight
Create and submit an outline for the video and essay below. The outlines should be short restatements of the arguments presented in the video and in the essay.  In the essay, which of the two points of view, argued for by the video and article below, convinces you more?  Among your list of societal/political worries about the future, does this make the top 5? top 10? Submit to the homework server in .doc, .docx, or google-doc form.
 
1. "Humans Need Not Apply" video.
2. The Economist (magazine) has an excellent, though long, introduction to technological unemployment issues. I'd like you to read the section on The Impact on Jobs (pp. 5 - 8)

Be ready to contrast the article with the video in class. 

The rest of this long Economist essay is also very good. I urge, though don't require, you to read the other sections.
TicTacToe homework #3, due Wed, 5/29 midnight Building a real competitor's engine.  Here are the specs for version #3.
Ethics of surveillance One example to know about in the debates:  Chinese surveillance of Uighurs in Xinjiang province.
Ivan Galakhov invites y'all to... ...the Stuyvesant Integration Bee, for all you calcish Anthophila.
Your own web server!  Not yet -- I'm working on a better version.
Basic TicTacToe structure -- homework #2, due Mon Here's HW TTT#2.  Use CoLab or your own version of Jupyter notebooks.
TicTacToe calculations due Wed 5/15 midnight 5/20 Calculate the 7 requested numbers in TicTacToe games and put them into the Comments-to-Teacher.
Presentation: AI in the military In period 10, Raunak Chowdhury and Jonathan Singer will present on AI and visual targeting systems in the military.  Both classes should view and read these slides.  Period 10: be prepared to discuss what limitations you would propose on the technology presented in the slides.
Perceptrons, for Thurs, 5/9 - class
  • We covered the use of a this perceptron to calculate the effect of an AND gate on a pair of digital (logical) inputs. 
  • We used this diagram in our explanation for to show that a line can separate the (1,1) input from all the other inputs, and therefore the equation of that line can supply the weights and bias to implement the AND function. 
  • For the AND function, the weights were: w(1)11 = w(1)12 = 1 and b(1) = -1.5. 
  • We also found that by moving the dividing line down, we could separate the input (0,0) at the origin from the other 3 inputs, to accomplish an OR function, with the weights being w(1)11 = w(1)12 = 1 and b(1) = -0.5
  • Using this dividing line argument, we found that the XOR function could not be created by a single-layer perceptron because no dividing line could separate the inputs (1,0) and (0,1) from the other two inputs.

Homework/classwork:

  1. Convince yourself that there are 16 different possible logical functions (logic gates) with two logical inputs.  Don't forget the function that outputs 1 for every input, and the one that outputs 0 for every input.
  2. Convince yourself that, of the 16 possibilities, only 2 of them cannot be accomplished with single-layer perceptrons, with suitable weights and biases.  One of them is, of course, XOR.  What's the other?
  3. We can construct an XOR function using a neural net consisting of hidden layer of 2 neurons. Find the appropriate weights (6 of them) and biases (3 of them) to accomplish the XOR.  Note: there is not unique solution to this.  Given a lot of freedom, try to assign the minimum number of different values to the weights and biases (in other words, try to make as many of them the same as possible).
  4. Put you results into the Comments-to-Teacher in the homework slot.

Any of you should feel free to answer questions from other students, including coming up to the board to explain.  Just don't give the answers away directly.

Neural Networks: 3Blue1Brown videos

Calculations due Sat. night.

There are 4 excellent(!) Youtube videos on how neural networks work from 3Blue1Brown.  Watch at least the first one by the time we return, and more if possible.  Keep the following questions in mind:

  • How did he decide on the number of nodes in his network architecture: namely how many hidden layers and the number of nodes in each hidden layer?
  • How do you calculate the number of weights given the details (above) of the architecture?

  • Do the following calculation:
    • (Calc 1): if his network is presented with a single image, how many operations (each multiplication is an operation, as is an addition) does it take to calculate the single result (the list of 10 probabilities) from that image.  Ignore the calculation of the sigmoid function, and we're only looking for approximate values (+- 10%).
    • (Calc 2): Having done that, now calculate that for a run 30,000 different images. You've just now calculated one data point measuring how well the network classifies (initially, it classifies very poorly). 
    • (Calc 3): Assume that it takes 100,000 data points (trials) to train the network.  If each operation takes a 10 nanoseconds (a single core processor), how long does does training take?  (Ignore the work of backpropagation, if you know about that.)
  • Put the answers into the Comments-to-Teacher
Smarter Sudoku algorithms project, due Mon, 4/29 midnight
  • You'll create and submit a small report webpage for this project.
  • You can work in pairs.  If you do, the report should contain a short discussion of the partnership and division of labor/insight.  Both partners must submit to the homework server.
  • You need to create at least one, and preferably more than one smarter and faster algorithm. The smartitude of an algorithm smart-algo will be measured by the ratio of your backtracks(naive-algo) / backtracks(smart-algo) for each algo for each board.
  • See link below for my webpage format for your report.
  • Upload your webpage to your public_html directory, and put the link into the Comments-to-Teacher.  Include in the comments your discussion of the partnership and division of labor.  Also include whether you want your webpage to be seen by others.
  • Use (Sudoku-boards-Master.txt)

Here's the link page for results.

Wednesday's presentation from AI4Anyone.org Fill out the post-presentation survey here: https://bit.ly/aiforanyone2
Audrey Soo (of period 9) took notes on the presentation (thanks Audrey)
Sudoku smarter: heuristics It's time to get smarter, now that you've solved some boards.  Create a sequence of smarter algorithms and reduce the number of backtracks.  Preferably have an input parameter that chooses the level of smartitude so that you can compare algorithms on the same board.  Try these out on the easier boards that you've already solved.

Here (Sudoku-boards-Master.txt) are all of the Sudoku boards I have.  All of the ones that you haven't yet seen are harder than the ones you have.  Once more, solve them, and keep track of the number of backtracks.
extra credit: take Alexnder Radu's WordLadder program and speed it up. Here's his program (it works, BTW), but runs in 7.3 secs of CPU time.  When you download the file, change its suffix from ".txt" to ".py".
Your job, should you decide to accept it, Mr. Phelps, is to choose one modification to the program to speed it up the most (no, a complete rewrite is not one modification).
There's now a homework slot for submissions, submit it and run it against the Program Tester.  If you've made a substantial improvement in runtime, you'll get extra credit.
Constraint Satisfaction Problems -- Sudoku!

Naive version due Sun, 4/14 midnight
Here are a number of Sudoku boards that we'll be working with. 
Here's  -- some info on the coding for this problem.
Here's the main loop for my  naive (simplest) version

Here are my backtrack numbers for various boards (note that they're not strictly correlated with NYTimes's estimation of difficulty):
A1-1,Easy-NYTimes,unsolved: 126
A2-1,Medium-NYTimes,unsolved: 15,504
A3-1,Hard-NYTimes,unsolved: 6,395

Here are the instructions for program-tester.
Allowed imports:
import sys
import copy

Your program will be given 3 command-line arguments:
1) name of input file (which will contain a subset of Sudoku boards)
2) name of the output file
3) name of the board to solve: that will be one of the following:
3a) A1-1,Easy-NYTimes,unsolved
3b) A2-1,Medium-NYTimes,unsolved
3c) A3-1,Hard-NYTimes,unsolved

The output file should be just the 9 lines of the solved board -- just like in the file Sudoku boards.  Nothing else.

For the homework server submission:
The homework slot should contain more information, though.  I'd like you to run your program on all 3 of the boards named above (3a,3b,3c), and put into the Comments-to-Teacher the number of backtracks you've had for each.


The reality of simulations A Coffeehouse Conversation on the Turing Test, by Douglas Hofstadter  (you needn't read the Post Scriptum section at the end)
The Seventh Sally by Stanislaw Lem  (much shorter sci-fi short story)
Sudoku -- data Here is a sudoku data file (Sudoku-boards.txt), that you'll use for testing your Sudoku programs
History of AI, and particularly machine learning Read through these histories Timeline of AI. (esp. Pascal, Leibniz, Babbage, Boole, Russell & Whitehead, McCulloch & Pitts, von Neumann, and then the modern era).
and Timeline of Machine Learning (Bayes, Markov, Rosenblatt, Minsky & Papert [remember Netlogo?], Deep Blue, and then the modern era [2010+])
Calvin Lee's Machine Learning talk on finding and interpreting text in pictures.  Due Thurs, 3/28 Write a couple of paragraphs on the talk -- your impressions, what you learned.
The Word-neighborhood exploration project, due Sun, 3/31 midnight - Near the bottom of the Wordladders Homework, in the Challenge Problems section, I've included some ideas/questions for further exploration.  These were just a few of the ideas that occurred to me as I was creating that homework task.  
- The interconnectness of words is a fascinating area.  The word-to-word connection we've been using -- single char replacement -- is just one of many possibilities of character addition, deletion, replacement (and other, less minute connections, like anagrams, or even rhymes, homonyms, synonyms, etc.).  
- For instance, here are the results of a Washington Post contest.
- You and your partner should explore some ideas on word-connection, and come up with interesting results if you can.  You can also mention questions that you've explored that failed to blossom after research.
- Write a document about the questions and results (Google doc, MSWord, LibreOffice, pdf, txt) and submit it to the homework server. Both partners must submit their own copies.  Also, in the Comments-to-Teacher, indicate who your partner is.
Wordladders Homework, (due Fri, 3/22, midnight)  
The makeup test (15) is here.

Assignment due on the homework server on Fri 3/15 midnight.

Put comments on corrections into the Comments-to-Teacher and also upload the correct, running program.

Copy the answers you wrote on the particular test that was graded into Python and fix your answers until they work.
1. For the first question, use the 3 lines of data on the test as your test file, and see if you get the answers I gave on the test.
2. For the Dlist, you'll have to write both the insert() and delete() methods, as well as __init__() and tolist().  Then do the following:


def insert_all(the_dlist, the_input_list):
  for element in the_input_list:
    the_dlist.insert(element)

a = Dlist()
insert_all(a,[4,5,2,3,2,7])
print(a.tolist())

and you should get: [2,2,3,4,5,7]

The pre-Wordladder assignment, due: Wed, 3/13 midnight The assignment is here.  Here's wordlist file, dictall.txt.
Correction for period 9: As the assignment really says (rather than what I might have told you): if the words in the input file, are 4 chars long (hint: they will be), you'll need to create a dictionary of all 4-character words in "dictall.txt" and their neighborhoods, not just the words in the input file.
This will be tested by the program-tester, so submit your file to the homework server, then go to "View Homework" and test it.
Search techniques Here's the paper describing the Uninformed (breadth-first) search more precisely.  I'd like y'all to read this through, slowly, and follow each step in each of the examples.
File Reading and Writing on Colab If you want your Python notebook to read a datafile that you have, upload the file to the "Colab Notebooks" folder of your Google drive.  Then read the following about reading and writing files on Colab.
Tues.3/5: I'm out today.  Here's the word on Friday's test: Not so good.  A good number of you were not really prepared to write working/correct answers to those questions.  I'm going to give you another chance. I will give a make-up test on Friday.  The questions are going to be very similar, though not exactly the same.  Remember: a doubly-linked list not only keeps track of the first node but also the last one.  I would strongly recommend that you write and test code for questions 1 & 2 on  last Friday's test.  If you do so, you'll do weller (more bet) on this Friday's.

Also, test your priority queues with the code below.
Testing your priority queues Create two more, simple methods for your priority queues:
push_all(lst) -- pushes all of the elements of the provided list
internal_list() -- returns all of the valid elements of your internal list, except for the first element, which, presumably you're not using.  Note: we're checking the order of the elements in your internal storage list -- therefore you'll be simply returning a slice of your internal list.

Then: execute the following code:
p=PQueue()
p.push_all(list('PETERBR'[::-1]))
print(p.internal_list())
p.pop()
p.pop()
print(p.internal_list())

Answer should be:
['B', 'E', 'E', 'R', 'T', 'R', 'P']
['E', 'P', 'R', 'R', 'T']
Create a PriorityQueue, due Sun. 3/3 midnight. Create a PriorityQueue class using a list.  Here's the assignment.
AI Topics O'Reilly, the technical book publisher, has an interesting site on AI, full of AI ideas/trends.  This is pro-business, pro-AI.
Program Tester Docs  
Readings for the break
  1. The Turing Test in the original: Computing Machinery and Intelligence by Alan Turing -- this is (possibly) the most famous paper about AI.  Read sections 1, 2 and 6; the other sections are optional.
  2. AlphaZero: read the first two pages of the research paper announcing AlphaZero -- this is the type of publishable paper you'll find in AI research. BTW, for those of you who have Netflix accounts, there is a documentary there called ... "AlphaGo" about .... AlphaGo.
Terms you should know for Friday's quiz  
Rachel Ng's StackOverflow link to creating a PDF file from a Colab notebook A bit painful, but seems to work:  https://stackoverflow.com/questions/52588552/google-co-laboratory-notebook-pdf-download#answer-54191922
Second programming assignment, by Fri midnight. Using your BinTree to test the Log2() growth of searching using binary-trees.  Instructions here: BinTreeAssignment-2.html
You may now upload your programs (.py file) to the homework server, although the ProgramTester is not quite ready.  Once uploaded, the ".py" suffix of the file will be changed by the homework server to ".txt" on the homework server (for reasons you can ask me about). 

The Program Tester is now ready for you.  Submit your .py file to the homework slot, then go to "View Homework" and you'll see a "Test Program" button.  Try it.  If the program passes the test, it will say so.  If not, you'll get some error message.  If you can't decipher the error message, talk to me, or send me email.  If you can decipher it, then fix your program, upload and retest.  If you press the Test Program button and you don't get an answer with a couple of seconds, do not keep trying over and over again (not only does hitting your head against the wall repeatedly rearrange your personality, but it's also not good for the wall).
Read Machines That Think: Prologue and Chaps. 1,2 by classtime on Friday (2/15)  
And now for a word from our very-own Joan Chirinos...
Hey folks!
My name is Joan Chirinos, and I'm the president of the Stuy Competitive Computing Club.

"What's the Competitive Computing Club???"

Good question! As upperclassmen, we will teach you how to use your rad CS skills to solve the problems typically found in CS competitions. 
We will also use this knowledge in a number of fun activities, including mini-competitions! 
With your immense knowledge, you can even go to PClassic or SJC, larger and more structured CS competitions!

If this sounds interesting to you, please fill out the form below. I will be emailing you shortly about our first meeting!

https://goo.gl/forms/GvWdAK0RryQZDIm82 
Create a Binary Tree class.
Due: Mon. 2/11 midnight.

Here's the BinTree notebook to complete.  Once completed and tested, download the HTML version of your notebook and submit to the homework server.

Slight change in the notebook above: the tree should have no duplicates.

Here's my answer: BinTree

Sample Python classes Here's the Stack class (MyStack.ipynb) that I was developing in class -- and the HTML version.

Here is a Notebook version of a singly-linked ordered list, as well as the HTML version.
Jupyter notebook on reading files File_Reading.ipynb
Attend:

Mon. Feb. 11 (after school) talk in the cafeteria, 3:45-5:00.

Dr. Nikolaus Kriegeskorte is a professor of neuroscience and a Principal Investigator and Director of Cognitive Imaging at the Zuckerman Mind Brain Behavior Institute at Columbia University. He uses deep neural network models, a form of AI, to mirror the brain’s visual system. Over the years, Dr. Kriegeskorte and his team have been improving their systems to mimic biological processes with greater accuracy. Moreover, these AI models not only simulate vision system but also other types of intelligence, such as language, reasoning, and motor control.

Website:

https://zuckermaninstitute.columbia.edu/nikolaus-kriegeskorte-phd
Python 2 vs. 3  
Python Classes Here is some quick documentation on classes in Notebook form (so you can play with and modify them) and webpage form.
More Python Review Here's a second Jupyter Notebook Python-Review-2.  Download and study, along with its links.
Python review Here's a Jupyter notebook (right-click to download).  Load it into a working Jupyter server (at school, or at home, or Google's Colab) and:
- read all of my Python review docs (#2 and #3) in the first cell
- in your browser, bookmark the PQR reference
- I have left a cell below each review topic in the notebook for you to play in.  Play in them.
Richard Gruet's Python page This is my standard quick-reference for Python.  It's the first place I go to for a quick look at the operators and methods of built-in datatypes.  I often have this up in a different browser tab with programming.   Although it's only really accurate for v. 2.7.x, there is very little difference between this and 3.6.x (and anyway, the 3.6.x+ compiler will warn you if you have sinned).
First task: fill out your Profile on the Homework server.
  1. Go to the main class link page (http://bert.stuy.edu/pbrooks)
  2. Click on the Homework/Grade server link
  3. Choose the Profile menu item
  4. Choose your period and name
  5. Log in with your OSIS number as your password
  6. Fill out your email address and preferred first name only, DO NOT CHANGE YOUR PASSWORD YET.
  7. Save: (Change Profile button)...
  8. (optional, but recommended): now you can change your password for this Homework server, if you want.
  9. REMEMBER this password!
Help from Mr. Brooks Feel free to come and see me during periods 6, 7, or 8 in room 301 (just walk in) or by appointment beforehand just after school also in room 301.
Sending email to Mr. Brooks:
Send mail to: pbrooks@stuy.edu

You MUST include your name in the subject line or body of the message, otherwise I won't know who it's from.
Stuyvesant bell schedule  
Homework/grade server  
Support Wikipedia