Projects
trainML Project allow teams to share access to jobs, datasets, and models under a single billing account.
Projects
trainML Projects organize trainML resources
into groups that can be shared with others or can have access to different third-party
resources. All trainML accounts
have a Personal project that cannot be shared, modified, or removed. Additional
projects can be created and those projects can be shared with other users. All projects
have a single owner (the creator of the project) that can control user access to
the project and configure the project's access to third-party resources. All resource
charges (compute and storage) accrue to the
project owner (or their billing account,
if configured). Additional members to the project can have either read-only or full
access to the three different resource types: jobs, datasets, and models.
Only add users you trust and know personally as members to your project.
Creating Projects
The Projects Dashboard allows you create and manage the projects you own as well as view or leave projects that you are a member of. To create a new project, click the Create button. Enter a name for the new project and indicate if you wish to prepopulate this project with your Personal project third-party keys. Once the project is created, click on the name of the new project under My Projects to view the current project configuration.
Once created, projects maintain a separate list of third-party keys that can be configured in the project details page. Only the project owner can view and set these keys. By setting keys on a per-project basis, you can ensure each project has granular access to only the data and resources it needs.
Although only the project owner can view and set the third-party keys in the user interface, it is trivial for a project member with access to jobs to view the key material. Do not grant access to a project member that you would not grant access to the third-party resource.
Project Settings
If you have configured your own resources in CloudBender™, you can configure a project to only run on resources that you own. Not only will jobs only run on your resources, but datasets and models will be created and loaded only on your resources as well. To enable on a project, check the Use Only Owned Compute Resources box in the setting section and click Update. Once you make this change, click the Refresh Available GPU Types button to update the list of GPU types available to this project. This will prevent project users from accidentally selecting a GPU type that you do not own.
If you add more GPU types in CloudBender later (e.g. enabling a GCP region) and you do not see the new GPU types in the job form list, return to this section and click the Refresh Available GPU Types button again.
Managing Project Members
To share access to your project with another trainML user, click the Add Member button in the Project Members section. Enter the email address the new project member in the Email field and configure the type of access the user should have to the project. Be sure the email entered matches the email address the user created their trainML account with. Read-Only access allows the user to list the resources, view resource details, and download resource logs. All access allows the user to create, edit, and delete resources. Additionally, Read-Only job access allows the user to access and therefore edit the contents of a Notebook, but not create, start, stop, or remove one.
A common member configuration is to enable All access to jobs, but Read-Only access to datasets and models. This allows the member to use datasets and models as part of their jobs but not to create new ones or delete ones that exist. They would also be able to create new model versions by using the trainML output type for Training Jobs, but would not be able to remove or edit the model the job produced.
A member's access can be changed or removed from the member list using the Update and Remove button. Changes and removals are effective immediately.
Project members can also leave a project at any time, which will revoke their access to the project and remove them from the member list.
Using Projects
By default, all accounts use the Personal project to create and list resources. Once the account has access to additional projects, either by creating a project or being invited to one, the project must be selected to view or create resources for that project.
Web Interface
If an account has access to a non-personal project, the Project menu item will appear in the header bar. It will display the currently active project and allow you to select from the other projects you have access to. To change your active project, click on the project name from the header bar menu. Once the project is changed, the list of jobs, datasets, and models will update to show only those resources in that project. All new resources created while a project is active remain in that project. Resources cannot be copied or moved between projects, so be sure to have the correct project selected when creating jobs, datasets, or models.
CLI/SDK
All operations using the trainML SDK/CLI use the trainML account the API keys were generated from. If this account has access to non-personal projects, the SDK can use other project by configuring the active project. To set the active project, run the configure command:
trainml configure
This command will output the currently configured active project (UNSET defaults to Personal) and allows you to specify any project you have access to as the new active project.
Current Active Project: Personal
Select Active Project: (My Other Project, Personal, Project Shared With Me) [Personal]:
Once you select a project, it will store the results of your selection in the config.json file in the TRAINML_CONFIG_DIR folder (~/.trainml by default). Once the active project is set, all subsequent operations will use the selected project.
This setting can also be overridden at runtime using the environment variable TRAINML_PROJECT:
TRAINML_PROJECT=<PROJECT ID> python create_job.py
or by instantiating the trainML client with the project keyword argument:
from trainml import TrainML
trainml = TrainML(project="PROJECT ID")
await trainml.jobs.create(...)
You must specify the project ID (not name) when using the runtime options. The project ID can be found by running trainml project list.