Is there a way to turn off default accounts?

You can't turn off any default accounts because when activity or expense details are not provided by you, a default account is used to fill in the missing items. For example, if an expense item did not have an income account provided, the default expense account is used to provide that. You can, however, change the default account anytime in Settings > Integrations > Sync Settings.