Monday, August 14, 2023

Looking at the 'mind' of a Neural Network

This is an ongoing project, but since I've got a nicely trained convolutional neural network sitting around, I'm trying to figure out what's going on under the hood.  No one seems to know, so I figure that's a pretty good spot for me.

First trick is to gain access to the weights stored in a trained CNN.  Once I have these, I can 'display' them in various forms to see if I can pick out any discernible patterns that might lead me to a breakthrough.  The format of the input weight file took a little while to figure out, and I'm pretty sure I'm reading it in correctly.  The output 'image' might be where I'm currently going wrong.

What bothers me about what I'm currently seeing is that there are patterns that probably shouldn't be there.  Here's a good example:




I'm pretty sure the similarities in columns is indicative of a read/write problem.  I'll figure it out, but ugh.

No comments:

Post a Comment