Missing Notes in Sidebar and Tag Searches

[Breen]

Random notes are missing from the tag sidebar or when I search Bear for the tag (i.e., #/journal/tagname), but do show up when I search for the text string without the # symbol (i.e., /journal/tagname).

The problem exists on all my devices: Mac, iPad and iPhone (all running the latest Bear version 2.2.0).

I discovered this when I couldn’t locate a note that I knew I had written. I started digging and discovered that I had missing notes in five different tags in my system (out of about 40 tags).

I thought my Bear database had become corrupted somehow. I decided to nuke and pave my system:

1. Exported all my notes to a Markdown folder.
2. Deleted all my notes. I emptied the trash.
3. Synced the now blank database to the my other Mac, the two iPads and the iPhone, one device at a time.
4. Restarted every device.
5. Imported the Markdown folder and synced my devices, again one at a time.

After starting from scratch with a brand new notes database, the five tags with the missing notes were fixed. They properly showed all the notes in the sidebar and in tag searches. But now a different set of tags has missing notes in the sidebar and in searches — all this after a complete rebuild of my system.

As an illustration, I have 28 notes tagged with #journal/adventurelog in Bear, yet only four show up in the sidebar or in a search. A search through all my notes for journal/adventurelog (note the omitted hash tag symbol) retrieves all 28 notes.

Right now, three other tags in my newly rebuilt system have missing notes. The other 37 tags are perfectly fine. I compared the format and content of the tags in notes that show up vs. those that don’t. They’re identical. If I edit a missing note, it will show up in the sidebar and in searches. For whatever reason, Bear is choosing to simply the tag information from some notes — at random.

This is inexcusable behavior for a notes app, especially one that relies exclusively on tags. What makes this worse is that the problem is very difficult to detect — it’s hard to see what isn’t there, unless you somehow remember, which defeats the purpose in the first place.

Any ideas on why/how this happening?

[krssno]

Can you show a screenshot of the notes itself? One of a note that is shown and of a missing note.

Is the option “hide subtags” activated?

The very first thing I would do in your situation: take one of the missing notes, delete the #journal/adventurelog-tag and write it again by using the autocomplete field. Is the
Missing note shown after the change?

[Breen]

@krssno

Can you show a screenshot of the notes itself? One of a note that is shown and of a missing note.

Here’s a screenshot of a note that shows up in the sidebar:

Here’s a screenshot of another note that is missing from the sidebar:

@krssno

Is the option “hide subtags” activated?

No.

@krssno

The very first thing I would do in your situation: take one of the missing notes, delete the #journal/adventurelog-tag and write it again by using the autocomplete field. Is the
Missing note shown after the change?

At first, I had the same thought as you. Maybe there’s something wrong with the tag itself? But then I figured that if I make any edit at all to a missing note, it will suddenly reappear in the sidebar and in tag searches. So, it’s not the tag itself that is wonky.

[krssno]

@Breen

But then I figured that if I make any edit at all to a missing note, it will suddenly reappear in the sidebar and in tag searches

That seems weird.
That is odd but also seems to be the key point to understand what is going on. Sorry, I can’t help.

@matteo @trix180

[matteo]

@Breen

Any ideas on why/how this happening?

Hello! That shouldn’t happens and I’m sorry that you’re experiencing such a problem.

I’m not 100% sure on what’s going on, but we can’t rule database corruption, it’s unlikely, but just deleting from Bear doesn’t delete the database file so if that’s the case, the re-import you did would not help.

If you’re willing to help us debugging the issue, I’ll send you a DM with some more questions and things we can try to solve the problem, at worst I can manually rebuild the database and ensure that all data is correctly placed.

Let me know!

[Breen]

Please do follow up with more things to try. Thank you.

[apgold]

This probably isn’t related, but I’ll post here just it case it sparks something.

For the past several days I’ve been experiencing Shiny Frog cloud kit sync issues - but only on some devices, not all. Even though all devices are on the same network and running the latest version of Bear. My devices are a Mac Studio, MacBook Pro, iPhone, and iPad.

Starting earlier this week, notes I took on one device were not showing up on others. So not as subtle as your errors … in my case, any edits/creates from one device would not sync with any of the others. Again, all on the same network, all up-to-date.

I submitted a report to the Bear team and two days later they replied requested an error log, which I sent them. A few hours after sending the error log, all my devices started to properly sync. I have no idea if the Shiny Frog team did anything, though the timing seems awfully coincidental.

The log reports show multiple errors related to iCloud.net.shinyfrog.bear and “service unavailable”.

I should mention that I tried everything to fix the errors over the past few days to no avail, including rebooting each device, restarting Bear multiple times, ensuring Bear was up to date, and trying multiple different wifi networks. Nothing worked for several days until a few hours after capturing and sending error logs.

I’m not entirely sure how I feel right now. Pleased that my notes are now syncing? I suppose. But I’d feel a hell of a lot better if the Shiny Frog team said something like, “We found the problem and we fixed it” versus what I expect to hear back along the lines of “we didn’t do anything”. In which case, I’m quite concerned about the reliability of future notes.

All that to say: I’m wondering if the strangeness you’re seeing could be device-sync related in that some sort of corruption might be occurring during the sync. If you disable sync on your two iOS devices and just use your Mac, do you still see the tag search weirdness?

As I said, probably zero correlation with my issues, but tossing it out here just in case.

[matteo]

@apgold

The log reports show multiple errors related to iCloud.net.shinyfrog.bear and “service unavailable”.

This error message is generated by iCloud when there are server issues. Unfortunately, there’s little we can do since we are not Apple. Apple typically resolves these problems quickly, but some users might still experience downtime occasionally.

@apgold

But I’d feel a hell of a lot better if the Shiny Frog team said something like, “We found the problem and we fixed it” versus what I expect to hear back along the lines of “we didn’t do anything”

We have no control over iCloud and CloudKit; everything is managed by Apple. If we make an error, we can update our app to fix it. However, unless you see an update from us, there’s nothing we can do.

Bear is primarily a local app, meaning all your data is stored directly on your device. We use CloudKit solely to sync your data across multiple devices. Therefore, even if CloudKit experiences downtime, there is no risk of data loss

[apgold]

Thank you for the clarification @matteo - that is very helpful and comforting! That further suggests my issues are completely unrelated to this tag search strangeness. My apologies for clouding the issue.

[danmills7]

@matteo

Bear is primarily a local app, meaning all your data is stored directly on your device. We use CloudKit solely to sync your data across multiple devices. Therefore, even if CloudKit experiences downtime, there is no risk of data loss

@matteo does mean none of my data is stored in my personal iCloud? Is my data only as protected (from a backup standpoint) as the number of devices I have syncing my database?
Effectively, this would mean I have to have at least one Apple device logged in to iCloud in order to preserve my data…

[matteo]

@danmills7

@matteo does mean none of my data is stored in my personal iCloud?

Your data is securely stored both on your devices and in iCloud.

Bear’s philosophy is to ensure all your data is always available on your device. iCloud is considered just another device, which you can enable or disable without affecting the data on your other devices.

[Breen]

Following up to report a successful resolution to the problem I was having with notes not showing up in the tag sidebar and in tag searches.

After some troubleshooting, it appears my Bear database had become corrupted. I thought by deleting all my notes and resyncing devices that I had reset things, but this was not the case. @trix180 sent me instructions on how to completely reset the database, which involves deleting a hidden folder on the Mac, deleting everything in iCloud, deleting and reinstalling iOS apps, etc.

It’s been a week now since the database reset, and everything has been working fine. Zero issues. I’m considering this problem resolved.

Thanks for the terrific support!