b55e7a9a82
Previously in the HLO parser/lexer shapes were tokens which were identified using a complicated regular expression. This made augmenting the textual form of shape difficult such as would be necessary for dynamic shapes or tiling. To avoid ambiguity and other problems a couple changes were made to HLO textual form, as well as some related clean up:
(1) Do not redundantly print the shape inside of the constant HLO instruction's "operand" field. Previously, constant instructions we printed like:
S32[2,2] constant(S32[2,2] {{1,2},{3,4}})
Now this is printed as:
S32[2,2] constant({{1,2},{3,4}})
This avoids an ambiguity where the values of the literal can be misinterpreted as a layout. Also, the shape was printed inconsistently: only when the rank was greater than one.
(2) Remove ShapeUtil::ParseShapeString, replace with ParseShape function in hlo parser.
(3) Merge hlo_token.h into hlo_lexer.h. It is only used by the lexer and parser which include that file and avoids potential confusion with the token HLO type
(4) Fix b/112302613 by removing the unused Shape field in the sharding attribute of HLO text.
(5) As part of this change primitive element types are now keywords which simplifies parsing. The fallout is that a bunch of values in HLO text named "token" had to be renamed. Also, change the HLO name sanitizer to avoid these primitive type keywords.
PiperOrigin-RevId: 225546437
47 lines
1.7 KiB
C++
47 lines
1.7 KiB
C++
/* Copyright 2018 The TensorFlow Authors. All Rights Reserved.
|
|
|
|
Licensed under the Apache License, Version 2.0 (the "License");
|
|
you may not use this file except in compliance with the License.
|
|
You may obtain a copy of the License at
|
|
|
|
http://www.apache.org/licenses/LICENSE-2.0
|
|
|
|
Unless required by applicable law or agreed to in writing, software
|
|
distributed under the License is distributed on an "AS IS" BASIS,
|
|
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
See the License for the specific language governing permissions and
|
|
limitations under the License.
|
|
==============================================================================*/
|
|
|
|
#include "tensorflow/compiler/xla/primitive_util.h"
|
|
|
|
#include <numeric>
|
|
#include "tensorflow/compiler/xla/status_macros.h"
|
|
#include "tensorflow/compiler/xla/test.h"
|
|
#include "tensorflow/compiler/xla/test_helpers.h"
|
|
#include "tensorflow/compiler/xla/types.h"
|
|
#include "tensorflow/compiler/xla/util.h"
|
|
#include "tensorflow/compiler/xla/xla_data.pb.h"
|
|
|
|
namespace xla {
|
|
namespace {
|
|
|
|
TEST(PrimitiveUtilTest, StringToPrimitiveType) {
|
|
auto expect_ok_and_equal = [](const string& str, PrimitiveType expected) {
|
|
TF_ASSERT_OK_AND_ASSIGN(PrimitiveType actual,
|
|
primitive_util::StringToPrimitiveType(str));
|
|
EXPECT_EQ(expected, actual);
|
|
};
|
|
expect_ok_and_equal("f32", F32);
|
|
expect_ok_and_equal("tuple", TUPLE);
|
|
expect_ok_and_equal("pred", PRED);
|
|
expect_ok_and_equal("s32", S32);
|
|
|
|
EXPECT_IS_NOT_OK(primitive_util::StringToPrimitiveType("F32").status());
|
|
EXPECT_IS_NOT_OK(primitive_util::StringToPrimitiveType("Pred").status());
|
|
EXPECT_IS_NOT_OK(primitive_util::StringToPrimitiveType("preD").status());
|
|
}
|
|
|
|
} // namespace
|
|
} // namespace xla
|