I have been using Cloud ML and stackdriver logging service for a while. However, today there seems to be a problem and logs are not coming up as usual.
Before when issuing a job logs such as the ones under would come up.
info: Validating job requirements...
info: Waiting for job to be provisioned.
info: Setting up Tensorflow.
However, today there seem a problem and these logs are not coming up even though the job would eventually go from preparing to running.
Cannot think i changed something here. The only thing i did from last job till now is request more ML units from Google. Is this an internal issue or something might be broken from my side?
These logs are also not showing on client command line when using stream-logs.
Stackdriver logs (in general, not Cloud ML Engine logs, specifically) are not guaranteed to arrive immediately. Your logs are probably just delayed. Can you let us know if they don't show up within an hour? We apologize for the inconvenience.