I've been trying to make this setup to work and I need some help.
How can I isolate team's boards and work items so that members of one team cannot view & edit other team's boards and work items?
My department wants to use a single ADO project for the development of multiple systems. Reading thought Msft's documentation, it is even recommended to do so:
In general, we recommend that you use a single project to support your organization or enterprise. A single project minimizes the maintenance of administrative tasks and supports the most optimized / full-flexibility cross-link object experience.
Even if you have many teams working on hundreds of different applications and software projects, you can most easily manage them within a single project.
There is no concept of "team ownership" of workitems in Azure DevOps. Instead the each workitem belong to an Area Path. Area paths can then be added to one or multiple teams (see Add area path to team). The workitems belonging to Area Paths of a team is what is shown on the teams board.
In order to achieve what you describe you need to first ensure that each team has their own area paths that are not shared with any other team (there is no limitation against this in Azure Devops as is). Once there are no shared Area Paths you can set permissions on the area paths
Area path permissions let you grant or restrict access to edit or modify work items, test cases, or test plans assigned to those areas. You can restrict access to users or groups. You can also set permissions for who can add or modify areas or iterations for the project.
I would recommend to remove all permissions from the root area path so that no permissions are inherited and assign only View Workitems in this node
and Edit workitems in this node
to the team that is owning the area path. While it is technically possible to leave the root area permissions as is and deny the abovementioned permissions for all other teams instead, that quickly becomes an administrative overhead and source of error (for example, adding a new team would require all existing area paths to be updated in such a scenario)
I would still recommend evaluating the single project setup even though it is a recomendation from Microsot. Your strict boundaries that do not even allow read access between workitems voids many of the advantages of the single project setup unless you are setting up some hierarchial team structure where a management team has access into multiple area paths and thus can view an aggregated board or do cross team searches.
Another thing to consier when having multiple teams that essentially work in silos in the same project is how tags work. The tags are project wide and thus a tag taht is created by team A will show up in the list of tags also for Team B (in autocomplete fields etc), which can be confusing.