Commit Graph

187 Commits

Author SHA1 Message Date
Michael Case
c3b42c815c Automated rollback of commit 20f03388ac
PiperOrigin-RevId: 217636584
2018-10-17 20:49:29 -07:00
A. Unique TensorFlower
cc0cf49a0d Check for the presence of a Worker machine when reassigning hooks in distributed
training jobs.

PiperOrigin-RevId: 217407558
2018-10-16 16:25:58 -07:00
Katherine Wu
ad836327e4 Replacing legacy_init_op argument in SavedModelBuilder with main_op.
PiperOrigin-RevId: 217389035
2018-10-16 14:50:07 -07:00
Gunhan Gulsoy
20f03388ac Automated rollback of commit 0d054f2085
PiperOrigin-RevId: 216732038
2018-10-11 11:36:55 -07:00
Michael Case
0d054f2085 Have TensorFlow opensource code use tensorflow_estimator module if available.
PiperOrigin-RevId: 216706771
2018-10-11 09:30:12 -07:00
Jonathan Hseu
d37f771cc5 Move TPU variables to the TPU device in TPUStrategy.
PiperOrigin-RevId: 215027511
2018-09-28 18:45:56 -07:00
A. Unique TensorFlower
17d73444f3 Update hooks for distributed jobs with a master node, to ensure that
summaries are written at the correct interval for jobs with long-running
evaluations.

PiperOrigin-RevId: 214993119
2018-09-28 14:36:51 -07:00
Mark Daoust
1a834d3aa8 Remove @{} api_links and ban "@{}" from python and md files.
PiperOrigin-RevId: 214964988
2018-09-28 11:21:49 -07:00
Sourabh Bajaj
5b971c7eae Fix the eval hook to run the correct number of steps when using TPU strategy
PiperOrigin-RevId: 214709465
2018-09-26 19:20:27 -07:00
A. Unique TensorFlower
69650fff2b Automated rollback of commit 844074c2a8
PiperOrigin-RevId: 214693201
2018-09-26 16:53:37 -07:00
A. Unique TensorFlower
844074c2a8 Update hooks for distributed jobs with a master node, to ensure that
summaries are written at the correct interval for jobs with long-running
evaluations.

PiperOrigin-RevId: 214678483
2018-09-26 15:28:19 -07:00
Karmel Allison
7af35581b6 Fixes typo in docstring to correctly name the new Estimator method export_saved_model
PiperOrigin-RevId: 214325271
2018-09-24 14:02:35 -07:00
Pavithra Vijay
867449616a Convert the new metric instances to (value_op, update_op) tuple in the EstimatorSpec.
PiperOrigin-RevId: 213548081
2018-09-18 17:27:14 -07:00
Mustafa Ispir
6c8f6920e8 Updates documentation of Estimator.predict to note that an issue with yielding and graph context.
PiperOrigin-RevId: 213528782
2018-09-18 15:36:59 -07:00
wangsiyu
72359f9cfa fix bug of lacking axis when using array.ops.concat in unwrap_and_concat function 2018-09-15 16:02:22 +08:00
A. Unique TensorFlower
1c2a300d48 Fix spelling in error message
PiperOrigin-RevId: 213062112
2018-09-14 16:28:00 -07:00
Eddie Zhou
47b1af2a3a Expose an axis argument for VocabInfo, which allows for warm-starting of the second axis of Tensors through tf.train.warm_start. Note that the underlying initializer already has this functionality (for example, for output layers).
PiperOrigin-RevId: 211709879
2018-09-05 15:42:44 -07:00
Yuefeng Zhou
9d76f6b2d0 Remove per-tower ready op since concat doesn't have a GPU kernel for DT_STRING.
The current implementation queries the global collection for ready op. Therefore there is no need to have a per-tower ready op.

PiperOrigin-RevId: 211187544
2018-08-31 23:08:23 -07:00
Katherine Wu
cda5ea80b8 Roll forward of commit 069f808e5c (rolling back rollback cl/210656847).
Fixing reference to _get_export_outputs_for_spec in TFMA (This function was refactored out, so the string has been removed from the list of methods that are copied from core Estimator).

*** Original change description ***

Automated rollback of commit 069f808e5c

PiperOrigin-RevId: 211122893
2018-08-31 12:12:17 -07:00
Pavithra Vijay
0a8aefbcdf Add support for the new metrics in the Estimator module.
PiperOrigin-RevId: 211015414
2018-08-30 18:10:15 -07:00
Yuefeng Zhou
007c234dfc Unwrapping per_device hooks in estimator.
PiperOrigin-RevId: 210984489
2018-08-30 14:58:24 -07:00
Katherine Wu
30a6b56176 Automated rollback of commit 069f808e5c
PiperOrigin-RevId: 210656847
2018-08-28 20:53:28 -07:00
Katherine Wu
069f808e5c Export Keras model to SavedModel.
PiperOrigin-RevId: 210648154
2018-08-28 19:04:24 -07:00
Karmel Allison
00045099ee Add estimator.export_saved_model without removing export_savedmodel in order to stage TF 2.0 removal of export_savedmodel. The new export_saved_model will not have the option to strip_default_attrs; the new behavior is that default attributes are always stripped.
PiperOrigin-RevId: 210559617
2018-08-28 10:10:48 -07:00
Priya Gupta
599bb5522d Handle the case where input_fn dataset may not contain labels.
PiperOrigin-RevId: 210436691
2018-08-27 15:00:40 -07:00
Yuefeng Zhou
524f931fa9 Only create steps_per_run_variable if it is tpu strategy.
PiperOrigin-RevId: 210187464
2018-08-24 18:10:39 -07:00
Aurelien Geron
0c5683c50b Add more doc for the dict returned by an estimator's evaluate() method in the api doc 2018-08-21 10:31:29 +02:00
A. Unique TensorFlower
6a10ee6979 Clarified some docstrings, fixed some formatting.
Also removed some dead code.

PiperOrigin-RevId: 209380727
2018-08-20 01:54:33 -07:00
Priya Gupta
3b7429d7b3 Enable evaluation in Estimator with TPU Strategy
PiperOrigin-RevId: 209209370
2018-08-17 14:48:16 -07:00
Mustafa Ispir
3cbcb1c751 Generates an actionable/useful error if train is called in wrong configuration.
PiperOrigin-RevId: 209061838
2018-08-16 15:41:51 -07:00
Mark Daoust
e4371880b1 Remove magic-doc-links from code.
This change contains no code changes. Only doc-strings.

We can't use relative links in code files, so we don't have much choice but to link to tensorflow.org/

The deleted links were to docs that no longer exist.

PiperOrigin-RevId: 209019572
2018-08-16 12:10:03 -07:00
Sourabh Bajaj
ff6525acf1 Step_fn should be able to receive un-wrapped inputs
PiperOrigin-RevId: 208929959
2018-08-15 20:57:39 -07:00
Jianwei Xie
fabf7a5c18 Remove unused Estimator overrides.
PiperOrigin-RevId: 208834294
2018-08-15 09:50:21 -07:00
TensorFlower Gardener
9299a96a2b Merge pull request from olicht:patch-1
PiperOrigin-RevId: 208493257
2018-08-13 09:38:49 -07:00
Sourabh Bajaj
054b3961f7 Create new MultiStepStopHook for running mutiple steps per run when using TPU Distribution Strategy with Estimator.
PiperOrigin-RevId: 208310915
2018-08-10 21:35:14 -07:00
Katherine Wu
689fecc071 Refactor out path concatenation in the SavedModelBuilder.
PiperOrigin-RevId: 208094190
2018-08-09 12:46:57 -07:00
A. Unique TensorFlower
611edc8c51 Support writing np.ndarray as tensors in Summary proto.
PiperOrigin-RevId: 208084201
2018-08-09 11:43:28 -07:00
Priya Gupta
1ca9571fde Distribute input in estimator correctly for eval.
PiperOrigin-RevId: 208070489
2018-08-09 10:37:18 -07:00
Mark Daoust
f40a875355 Remove usage of magic-api-link syntax from source files.
Back-ticks are now converted to links in the api_docs generator. With the new docs repo we're moving to simplify the docs pipeline, and make everything more readable.

By doing this we no longer get test failures for symbols that don't exist (`tf.does_not_exist`  will not get a link).

There is also no way, not to set custom link text. That's okay.

This is the result of the following regex replacement (+ a couple of manual edits.):

re: @\{([^$].*?)(\$.+?)?}
sub: `\1`

Which does the following replacements:

"@{tf.symbol}" --> "`tf.symbol`"
"@{tf.symbol$link_text}" --> "`tf.symbol`"

PiperOrigin-RevId: 208042358
2018-08-09 07:08:30 -07:00
Priya Gupta
96a77055bd Add an API to distribution strategy that allows running N steps. Implement this for MirroredStrategy and OneDeviceStrategy. Implemented in TPUStrategy earlier.
PiperOrigin-RevId: 207961939
2018-08-08 16:08:36 -07:00
A. Unique TensorFlower
e8adf96abc Fixed typo in Estimator documentation.
PiperOrigin-RevId: 207672785
2018-08-07 00:47:08 -07:00
Allen Lavoie
1bf206bc82 Split checkpoint management utility functions out of saver.py
Pure refactor, in preparation for adding a higher level checkpoint management utility. This utility will also need to work with the Checkpoint proto, and globbing it on to saver.py seems dirty.

PiperOrigin-RevId: 207179646
2018-08-02 15:51:17 -07:00
Sourabh Bajaj
d07564b2a4 Output the PerDevice variables from tpu_result and reduce them correctly on the host to yield a single scaler value from the reduce function.
PiperOrigin-RevId: 206990072
2018-08-01 14:17:58 -07:00
Priya Gupta
ffc9be66bc Support distribution strategies in Estimator.evaluate.
PiperOrigin-RevId: 206864512
2018-07-31 18:45:54 -07:00
olicht
a4c4dca8d0
Improve estimator.py doc-strings
* Fixed typos
* Warped python terms with ` ...`
* A lot of tf terms are now links to their docs
2018-07-31 22:10:27 +03:00
Zhenyu Tan
86a573b90c fix bug in estimator constructor when model dir is empty
PiperOrigin-RevId: 206775613
2018-07-31 09:40:34 -07:00
Zhenyu Tan
ee6d76a4a4 abstract config overwrite logic as public API
PiperOrigin-RevId: 206621941
2018-07-30 12:02:03 -07:00
Katherine Wu
9671996af3 Add estimator in contrib that loads its model function from a SavedModel.
PiperOrigin-RevId: 206048542
2018-07-25 13:54:23 -07:00
Nick Felt
26e531ee70 Automated rollback of commit 568727eed1
PiperOrigin-RevId: 205875586
2018-07-24 12:57:37 -07:00
Nick Felt
568727eed1 Add v2 summary support to Estimator.train() and MonitoredSession hooks
This change makes Estimator.train() support v2 summaries (tf.contrib.summary.*) out-of-the-box, to match the support for v1 summaries.  Estimator.train() will now handle the boilerplate necessary to initialize a file writer and enable summary writing every N steps, and will ensure that its own automatically exported summaries (for loss and global_step/sec) get written to the same underlying events file.

As part of this change, tf.train.SummarySaverHook, tf.train.CheckpointSaverHook, tf.train.StepCounterHook, and tf.train.ProfilerHook have also been adapted to write summaries using the v2 summary system (via a compatibility layer), instead of using FileWriterCache.

A couple additional smaller changes are:
- the 'session' parameter to FileWriter() can now be a callable returning a tf.Session instance.
- the introduction of tf.contrib.summary.record_summaries_if() which takes a boolean tensor for direct control of tf.contrib.summary.should_record_summaries().
- EstimatorSpec.train_op, besides a tf.Operation, is now allowed to be any Tensor-equivalent object rather than just a tf.Tensor.

PiperOrigin-RevId: 205843986
2018-07-24 09:53:22 -07:00