Can AI Reason? A Cool Experience I Had with Claude

While some people have difficulty seeing the value in LLMs, others are trying to find new ways to create. To solve. To innovate.

Here’s my story of a “HOLY SH**, it can do that??”

I’ve been using ChatGPT for about a year or so, mainly for its ability to search. I’m always skeptical of its output, and have MANY times called it out for lying or fictionalizing results that just wasn’t there. I have been on the “cautiously optimistic” view of AI powered LLMs.

The problem is, these AI powered LLMs couldn’t DO anything for me. It was always FINDING things. I found it a useful “Google 2.0”. Searching small stupid facts I was interested in. Writing up or making my language or document structure just a little bit better, things like that.

The next leap I had with the AI & LLM journey was when the Unsupervised Learning community recommended I try out Claude.

They were right.

I’ve been setting it up, trying to figure out how it works. Frankly just overwhelmed with it. There’s so much I want it to do, so much context I could provide it to create better results that work better for me. Especially with the Model Context Protocol that can really start bringing my own context and content to LLM results, without having to respecify the projects I was working on, the method I want it to answer in, or type of details I’m looking for.

Now, I use Notion a lot for my main thoughts, just the way it’s structured works for me.

Today, I connected an MCP server to Claude to see how it works. It was a small thing, but a huge win.

I was testing its features out, it could read my documents by just asking me “Summarize the blog I wrote today.”

The Problem - Error on Write

The problem is, I am having a problem with Notion & Claude being able to read and write to a table document I created in Notion. I was trying to create a document with all the tools I use on a daily basis as well as research around how to set up MCP servers with each of the tools. I asked it to read the page I created and add any research it could find on MCP servers that were available for those tools.

It was struggling with updating the findings into the Notion page. It kept erroring on write. I pointed this out and it tried a few options, before, without any push from me, decided it wasn’t able to create the page and instead created a page in the root directory.

Yeah, this was a bummer, but sure, the cool thing here was:

• It figured out it had an error.

• It tried a few other options to fix the error.

• Decided it couldn’t fix the error and wrote the page in a place it COULD write the document.

I then pushed it a bit further to figure out why it couldn’t create the page.

It then tested a few other ideas, and provided some possible reasons for the failure. Still no success, but we are making progress. It is trying to figure this problem out for me, which to me, is a great win.

While some may say “oh you’re being lazy, why don’t you just figure this out. You’re handing off logical problems to a computer and are going to be lazier because of it.” I don’t care about this problem. I care about what I’m trying to achieve now. This is a side quest that is stopping me from producing what I was trying to complete. I’ll gladly give this over to a computer to fix for me if its capable.

Anyway, I digress. On with the story.

See, Notion has two different ways to structure databases, and this may have caused problems for Claude and the MCP to interact with it.

• Database pages as regular subpages - These appear as clickable subpages nested under a parent page, essentially acting like a traditional page hierarchy

• Database pages as database entries - These pages exist as rows within the database structure itself, where each row/entry has its own dedicated page

I asked it to take a look at another table I have that is set up like option 1, see if that was causing the issue.

It still wasn’t able to create the page successfully, but I pressed on, at least I was able to see that wasn’t the issue.

Next I asked it to look if there was any reported failures online. It did find some issues with this MCP server I was using.

It also included a list of workarounds as well as which ones we had tried.

Later today, I asked it to check if there was any updates that would make it work again, it suggested adding some tags to the MCP config file, I did this and had it try writing to the file again, and wasn’t able to work.

But we made progress!

Is AI Reasoning?

There are many definitions and sub-definitions of what “reasoning” can be classified as, but here’s my take:

• It successfully navigated the problem

• It tried various different solutions, directly in my notion document, given the right prompting

• It came to the conclusion that there is an issue with the tool that it can’t fix, provided me a temporary solution and gave me some other options to make it work

While I’m sure ChatGPT is coming close to this level of interconnectivity, Claude definitely feels a little bit more active in the workflow, being able to access, write, test and DO.

Whether or not its reasoning, and whether or not other AI LLMs can do the same may yet be to be determined, but the progress here was truly a “holy sh**” moment I thought was really cool.

Celebrating the small wins sometimes should just as important as celebrating the big wins.